Python socket bind in a rootless container/world

My original task was to create a Python script to send ICMP echo-request (aka. ping) messages to some hosts and then do some basic reporting. This is a simple task right? Well, not really, at least, if you care about security. To be able to ping a host, we need to open a socket, which cannot be done by a regular user, either on a Windows nor on a Linux machine. Now I will concentrate on Linux systems.

The internet is full of with fantastic ideas, but sometimes it is maybe a better idea to think a little before you copy something from stackoverflow. In a bunch of topics, they advice you to run your script as root, because what can go wrong, right? Or disable SELinux, what could go wrong? Well, say no to these! (This can apply to other topics too.. don’t always accept the first solution, think, think, think.)

So my idea is to create a simple script in Python. I wanted to use a full Python and async implementation, so I choose the aioping (https://github.com/M-o-a-T/aioping) library. They state that the library can only be used, if you run it as root. This is not really acceptable, so let’s see, what can be done to solve this issue..

Let’s see, what’s going on under the hood. In the core of the library, it tries to open a raw socket:

import socket
socket = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.getprotobyname("icmp"))
socket.bind(('', 0))

If you try to run just this, as a non-root user, you will get the following error message:

bash-4.2$ python3.6
Python 3.6.7 (default, Dec  5 2018, 15:02:05)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.getprotobyname("icmp"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.6/socket.py", line 144, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
PermissionError: [Errno 1] Operation not permitted

Well, nobody lied, it really need to be ran as ‘root’ (well.. yeah no, maybe). But that sucks, so what options do we have here? We can use the magic of setuid/setgid or from kernel version 2.2, we can use Linux kernel capabilities. Next we will check these.

SETUID/SETGID

setuid and setgid (short for “set user ID” and “set group ID”)[1] are Unix access rights flags that allow users to run an executable with the permissions of the executable’s owner or group respectively and to change behaviour in directories.

https://en.wikipedia.org/wiki/Setuid

So we can just simply set the owner of the script to be “root” and then enable the setuid bit and bamm we are done. What an awful idea! Security folks probably started to heavily breath now! But they are right, don’t use that.

Linux kernel capabilities

For the purpose of performing permission checks, traditional UNIX implementations distinguish two categories of processes: privileged processes (whose effective user ID is 0, referred to as superuser or root), and unprivileged processes (whose effective UID is nonzero). Privileged processes bypass all kernel permission checks, while unprivileged processes are subject to full permission checking based on the process’s credentials (usually: effective UID, effective GID, and supplementary group list).

Starting with kernel 2.2, Linux divides the privileges traditionally associated with superuser into distinct units, known as capabilities, which can be independently enabled and disabled. Capabilities are a per-thread attribute.

http://man7.org/linux/man-pages/man7/capabilities.7.htm

This sounds good, let’s see what capabilities are out there and which can be interesting for us! You can find all the capabilities and details in the manual, here I collected the most interesting for this topic:

CAP_NET_ADMIN

Perform various network-related operations:
– interface configuration;
– administration of IP firewall, masquerading, and accounting;
– modify routing tables;
– bind to any address for transparent proxying;
– set type-of-service (TOS)
– clear driver statistics;
– set promiscuous mode;
– enabling multicasting;
– use setsockopt(2) to set the following socket options:
– SO_DEBUG, SO_MARK, SO_PRIORITY (for a priority outside the range 0 to 6), SO_RCVBUFFORCE, and SO_SNDBUFFORCE.

http://man7.org/linux/man-pages/man7/capabilities.7.htm

CAP_NET_BIND_SERVICE

Bind a socket to Internet domain privileged ports (portnumbers less than 1024).

http://man7.org/linux/man-pages/man7/capabilities.7.htm

CAP_NET_RAW

Use RAW and PACKET sockets;
bind to any address for transparent proxying.

http://man7.org/linux/man-pages/man7/capabilities.7.htm

Okay, so now we know, that in the Linux world, there’s no more only root or non-root, we can use these capabilities, so we can grant permissions more granular. But if we follow the “principle of least privilege”, we can ask: do we need to allow all these capabilities? Well no, not really.

If we add CAP_NET_ADMIN capability to a process, it will be able to do various things to our networking stack, like interface configuration, change of firewall settings and so on. We clearly don’t want this, and if we check the list of permissions, we don’t need them.

On the other hand, CAP_NET_BIND_SERVICE allows the process to bind on a privileges port (any port under 1024). We also don’t need this, so we can just exclude this capability as well.

The last thing is CAP_NET_RAW. This allows exactly what we need: allows the process to use RAW sockets and to bind to any IP address. Perfect, just what we need.

Linux capability sets

Now we know, that we need the CAP_NET_RAW capability. But how should we add this to our process? We can me these permitted, inheritable and effective (and many more, but these are the most important now).

Each thread has the following capability sets containing zero or more of the above capabilities:

Permitted
This is a limiting superset for the effective capabilities that the thread may assume. It is also a limiting superset for the capabilities that may be added to the inheritable set by a thread that does not have the CAP_SETPCAP capability in its effective set.

Inheritable
This is a set of capabilities preserved across an execve(2). Inheritable capabilities remain inheritable when executing any program, and inheritable capabilities are added to the permitted set when executing a program that has the corresponding bits set in the file inheritable set.

Effective
This is the set of capabilities used by the kernel to perform permission checks for the thread.

http://man7.org/linux/man-pages/man7/capabilities.7.html

Add Linux capabilities to a process

Adding Linux capabilities to a process is fairly easy with the use of the setcap command. As this is going to be a Python script, we need to add it to the Python interpreter. Be careful, if you add this to the global interpreter of your system, all scripts will have these capabilities! To be on the safe side, it’s worth to create a virtualenv, and only pimp that interpreter!

(+ means, that we add the capability, e=effective, p=permitted)

setcap cap_net_raw+ep /path/to/your/interpreter

We can check the capabilities with the getcap command (this is an example of the ping command):

getcap /usr/bin/ping
/usr/bin/ping = cap_net_admin,cap_net_raw+p

And that’s it. Now you should be able to run your script without running it as root. It has now more permissions, than a regular process, but still not running as root!

Running in a rootless Docker container

Okay, now we can run that thing without root. So now let’s put it in a container! If we run a Docker container, by default, it drops a bunch of capabilities, as we wont need most of them. In this case, we will only need the cap_net_raw capability, so we can drop all others.

To read more about capabilities: https://www.redhat.com/en/blog/secure-your-containers-one-weird-trick

Example command to run the container with only the net_raw capability:

docker run -it --cap-drop=all --cap-add=net_raw <image>

With the –cap=drop=all command, we can drop all the capabilities. With the –cap-add=<name>, we can add capabilities.

That’s it.

DNS based ad/tracking blocking setup with NextDNS (at home)

In the past I played with AdBlock, pi-hole, now with NextDNS. In this article, I will show you the major differences, then at the end I will show a step-by-step guide for installing and setting up dnsmasq+netxdns client (so you can skip the first part, if you just need the install guide 🙂 ).

Why DNS based ad and tracking blocking?

  • Using browser plugins might get pain in the ass, as sites can detect if you use it and make you disable it.
  • Sites can load slower, as your browser needs to deal with the not needed ads as well
  • Your internet usage is higher when you load additional ads
  • Companies can track, what sites you visit and show ads based on that (Facebook, Google, etc…)

What solutions can you use?

There are several options with which you can do DNS based ad-blocking:

  • Add lists to your hosts file (client or server)
  • Use an on-premis service like pi-hole
  • Use a cloud service like NextDNS
  • Use a hybrid solution: on-premis server + cloud provider

Add lists to your hosts file (client or server)

This is the most simple solution, there are bunch of lists on the internet, basically these are key-value pairs. If the name is in the list, then your host will resolve it to it’s local address, so your browser will not be able to reach that resource.

ProsCons
You can set it up without a server in your networkHard to keep it updated, have to mess with it
Provider independentWill only do ad blocking on the local machine
Works on all platforms (Windows, Linux, Mac)

Set up an on-prem system (like pi-hole)

When you have multiple hosts in your home (and probably you have: PC, Laptop, Smartphones, TVs, Home automation tools, etc..), you might want to have one centralized server for your DNS filtering. Pi-hole is a great solution for this, it was originally created for Raspberry Pis, which are small boards, on which you can run Linux (like a small home server). You can easily install it on bare-metal or via Docker.

Once you installed it, you can load any lists, what you can find on the internet.

ProsCons
You have total control over it, as it’s running on your deviceYou have to know, how to manage Linux
It’s free to useYou have to buy the hardware
You can easily manage pi-hole via it’s admin interfaceYou have to keep it running, update it, manage it, etc..
Most security/privacy, as it’s running on your deviceYou can only reach it from outside of your home via a VPN

For a security expert, this would be a great solution, as you have total control in your hands. However it gets messy, when you want to make your phone/tablet to use it, when your are out from home, as you have to use a VPN to stay secure.

Using a cloud provider like NextDNS

Using a cloud provider can make your life much easier, you just need to set their DNS servers and that’s it. NextDNS even has some client tools (which.. are really not necessary, but convenient for the masses).

ProsCons
You don’t need any technical knowledgeYou loose privacy at one point, as this service is managed by “someone else”
You can easily manage it via their web-adminYou might need to pay for it
You can also set it up on your smartphone, tablet, pcYou can’t really set it up on your no-to-smart smart devices, like TVs, Home automation tools

Using a hybrid solution with an on-primes server and cloud provider

If you have a little Linux knowledge and you have a home server/Raspberry Pi or anything, which is capable running Linux, you can easily set up a local server, then you can advertise it’s local IP address via DHCP from your home router.

Now I will show the steps needed for setting up dnsmasq and netxdns cli on a Linux box.

  • dnsmasq is a dns server, we will use it for caching, as once we have the resolved DNS address, then we don’t want to go to the cloud provider anymore (so we maybe need to pay less 😉 )
  • nextdns cli is a simple cli app created by NextDNS, it can act as a DoH (DNS over HTTPs) proxy, so your DNS traffic will be encrypted towards the internet but will be regular non-encrypted on the local network.

Setting up NextDNS cli

NextDNS cli is an open-source project, it’s managed by NextDNS. The repository can be found on github: https://github.com/nextdns/nextdns

The installation is easy, this will add the NextDNS repo to your repo manager (yum, apt, etc), the list of supported systems is reachable here: https://github.com/nextdns/nextdns/wiki

sh -c 'sh -c "$(curl -sL https://nextdns.io/install)"' 

The installation is a next-next-finish script, it will ask for basic things like you NextDNS ID. It will create a basic configuration, so you can edit it later. It will also make the nextdns client to start on startup of the system.

After installing it, we need to configure it to listen on the local address of the server (localhost/127.0.0.1). To do this, the setup-router must be set to false! (It’s not for this use anyways..)

/etc/nextdns.conf

log-queries true
report-client-info true
detect-captive-portals false
hardened-privacy true
timeout 5s
setup-router false
auto-activate false
listen 127.0.0.1:53
config <REMOVED>
bogus-priv true
use-hosts false

After changing the settings, restart the service

service nextdns restart

And check the logs, if everything is fine

nextdns log

Ad this point, nextdns is listen on 127.0.0.1:53, so non of your clients should be able to reach it. But this is fine. To check, you can use netstat:

netstat -dnlp | grep nextdns
tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN 3762/nextdns
udp 0 0 127.0.0.1:53 0.0.0.0:* 3762/nextdns

Setting up dnsmasq

Dnsmasq is part of most of the basic repositories, so just try to install it (if it’s not found, then you need to find and add the proper repo)

yum install dnsmasq

Add the following to the dnsmasq configuration (/etc/dnsmasq.conf)

domain=local.local
resolv-file=/etc/resolv.dnsmasq
min-port=4096
cache-size=10000
listen-address=<IP of your server>
bind-interfaces
port=53

In the resolv-file, we will tell dnsmasq, to use our nextdns client as the upstream DNS server. The listen-address must be the address of your server (like 192.168.X.X or 10.X.X.X etc)

Add the server to the resolve file (/etc/resolv.dnsmasq)

nameserver 127.0.0.1

This means, that dnsmasq will be the only, who can directly acccess the nextdns client.

Restart dnsmasq and make it start on system startup

service dnsmasq restart
systemctl enable dnsmasq

To verify, use netstat again:

netstat -dnlp | grep 53
tcp 0 0 127.0.0.1:53   0.0.0.0:*           LISTEN      3762/nextdns
tcp 0 0 <IP>:53        0.0.0.0:*           LISTEN      3589/dnsmasq
udp 0 0 127.0.0.1:53   0.0.0.0:*                       3762/nextdns
udp 0 0 <IP>:53        0.0.0.0:*                       3589/dnsmasq

Enable your new service on all your hosts in the network

Now your server is ready to take requests. You need to add the server’s IP address to the DHCP server settings of your home router, so it will tell the clients to use dnsmasq.

Setting up this can be different among different brands, so please contact your router’s manual if you need help.

Is it working?

To check, if dnsmasq is doing caching properly, you can execute the following on your server:

 pkill -USR1 dnsmasq

This will send a special signal to dnsmasq, so it will just put some statistics into the logs. If everything is fine, you should see something like this in /var/log/messages:

cat /var/log/messages | grep dnsmasq
[...]
dnsmasq[3589]: queries forwarded 9519, queries answered locally 1220
[...]

This shows, that around 10k queries were sent to nextdns client and around 1.3k queries were answered locally from the cache.

Conclusion

At this point, all your home systems are using NextDNS via your home server. Yes, even your coffee maker, so now more ads on that. But what’s with your Smartphone/Laptop/Tablet when you are not at home?

So now your queries are encrypted on the internet, and your ads are blocked. (You just gave up a little on privacy, as now NextDNS is the one, who knows everything 😉 )