Monday 10:50 a.m.–11:20 a.m.
File descriptors, Unix sockets and other POSIX wizardry
Christian Heimes
- Audience level:
- Intermediate
- Category:
- Python Internals
Description
Have you ever wondered how the OS manages open files and network connections, what this 'file descriptor' thing actually is all about, or what's so special about Unix sockets? In my talk I will give you a quick tour into the I/O layer and process model of Unix-like operating systems. You will learn how to securely identify and efficiently share resources between processes.
Abstract
So you think that file descriptors are just some numbers that refer to open files of a single process? Unix sockets are limited, ancient and useless relics from the era of mainframes and CGA monitors? In my talk I'm going to show you that there is more to these low-level implementation details than you might expect.
File descriptors play a fundamental role in the input/output API of POSIX compatible operating systems such as Linux and the BSD family. Every time data is read or written to disk, transmitted over network or exchanged between programs, file descriptors are involved. The core of Twisted's and asyncio's event loop are build around operating system functions, that can handle many file descriptors at once. The Python interpreter provides abstractions for file descriptors, such as file or socket objects. Sometimes file descriptors can leak into child processes, which not only lead to resource leaks but also to security holes.
Next up, you will get to know valuable features of Unix sockets. These local connections can harness security properties and authentication of the operating system itself. In fact it is even possible to share information across Docker containers and reliable identify the container id from a socket connection.
Finally I will show you some quick examples how file descriptors, Unix sockets and a couple of other POSIX APIs can be combined to implement efficient zero-copy exchange of data or process sandboxes like in the Chrome browser. These won't be ready-to-use recipes, but techniques and building blocks.
PyCon 2016 in Portland, OR is a production of the 