Projects/CCAPI for UNIX

This is an early stage project for MIT Kerberos. It is being fleshed out by its proponents. Feel free to help flesh out the details of this project. After the project is ready, it will be presented for review and approval.

The project proposal page should include the following elements:

Desired changes

Port the CCAPI currently used on the Mac, and being ported (at the time of this writing) to Windows, to UNIX.

It would provide some benefits over the current file-based cache:

• The user's keys don't get stored on disk, at least not in an easy-to-find place. With OS support, we could theoretically even avoid having them get written to swap space, or saved out to disk during a "suspend" operation, but those would be outside the scope of this porting project. We're also not aiming to do anything about the use of ptrace or /dev/mem or /proc to access the process memory of the CCAPI server. But these attacks are harder to carry out than simply grabbing a file from /tmp.
  • Counterargument: UNIX file permissions do give some level of protection, and there's still swap, ptrace, etc., in the general case of not getting special OS support.
• It's easier to enumerate the credential caches available to the user, which we can't trivially do with file caches quite so safely. This will be important for the Kerberos Identity Manager project that Alexandra is working on (cf. http://web.mit.edu/macdev/KfM/KerberosFramework/KerberosIdentityManagement/Documentation/html/kim_identity_overview.html ).
  • Counterargument: We could just look for all /tmp/krb5cc_* entries and return those for which lstat() reports a file owned by the user, but if an attacker can create a hard link to some other file on the file system owned by the user, he may be able to cause confusion depending on whether or not the file looks like a ccache file and how we handle either case. Checking for a link count of 1 helps only if the attacker can't make the file's other name go away, and if no automated cleanup job will do it either. A dedicated file system for credential caches would help, but we can't really require that. A dedicated directory per uid for credential caches, with write access only for the uid in question, would also do the trick, and we could implement such a thing.

It's also outside the scope of the porting project to support OS-specific IPC mechanism, like Solaris "doors", but it's intended that porting to such mechanisms should be easy should someone wish to contribute back code.

Functional Requirements

The server process must be able to work both as a per-user server, initially, and as a global server supporting all users, later on. (Windows will also eventually need the latter functionality.) The server process must be launchable independently at boot time or login time, and from the client library with minimal interference with the client process.

Design of implementation

Communications will be done via UNIX-domain sockets.

Most POSIX platforms that we care about currently (though there's no formal list of those) seem to support some variant of getpeereid(), which lets a process determine the effective uid of the process at the other end of a UNIX-domain socket connection. Obviously, since the socket is accessed through a file descriptor, it could potentially be open in multiple processes with different uids, in theory, but we'll assume that only one uid will be used on the client side, and either the user's uid or root on the server side.

On platforms without getpeereid or equivalent, some games can be played with access-protected sockets or files for both server and client processes to prove their identities to one another. This shouldn't be needed on our primary platforms, however, so initially the CCAPI support will simply be disabled on platforms without such support.

Some platforms support alternative IPC mechanisms that may have other advantages (e.g., Solaris "doors"); using these mechanisms is outside the scope of this project.

A server process can probably be launched from the client process krb5 library with fairly little impact on the process if the OS provides waitpid or wait4 routines; we can collect the exit status of the new process without accidentally grabbing exit status information on another process that the main application was waiting for. If the application is calling wait in another thread, though, it may collect the exit status on the launched server process, so the library code should be prepared to not get that information. The application may notice unexpected SIGCHLD delivery and exit status for a subprocess it didn't know about, but that's probably the minimum effect we can portably achieve.

The socket pathname will need to be hard-coded, as the CCAPI server (currently) has no dependence on the krb5 library, and thus no clean way to find the krb5.conf file and parse it.

The launch protocol will need to operate safely when multiple processes (perhaps under the same uid, perhaps under different uids) are attempting to launch the server at the same time.

The existing CCAPI server code will need updating to support per-uid cache collections. The initial connection protocol should allow the client to pass some sort of "session handle" string, to be used as a key to the cache collection in addition to the uid. On the client side, this session handle may be provided via an environment variable or local modifications to the code; the session identifier will not add strong insulation between sessions unless the local modifications are resistant to simple query (e.g., Ken Hornstein's per-session magic file descriptor could point to a deleted file with a 128-bit pseudorandom number, not easily accessible outside the session).

This session handle allows, but does not force, different login sessions to have different sets of credentials. By default, there will be no session handle, and all login sessions of a given uid will share credentials.

TBD: Should root be allowed to access the credentials of non-root users?

TBD: Some cleanup function may be needed to purge expired credentials for users no longer logged in, or who no longer have valid accounts on the system (e.g., Athena model of temporary accounts).

Tasks/milestones

• Implement per-user UNIX-domain socket listener launched from command line, and library code to connect to it, with getpeereid-type authentication checks. Get CCAPI's RPC code working with it. Test manually. ~1 week

• Implement launching of listener process from library. Test manually. ~.5 week

• Automated testing in dejagnu of these parts. (May require special configuration of the pathname for the UNIX-domain socket, to avoid requiring root privs.) ~.5 week

• Implement setuid, monolithic server launch process, including support for multiple cache collections identified by uid(+sessionid). Test manually. 1-2 weeks

• Automated testing in dejagnu of using already-running monolithic server, when dejagnu has root access. ~.5 week

Strawman protocols

Socket pathname for monolithic server is $localstatedir/run/ccapi.sock (localstatedir default is $prefix/var); check that directory and socket are owned by root. For per-user server, /tmp/ccapi-$uid.sock (possibly adding or ignoring the session id, so that initially we have only one cache collection); check that socket owner is user or root. Compile-time flags determine the mode used; possibly overridden by environment variables for testing, but environment variables must be ignored by setuid server launched by client library.

(Do we want a ccapi.conf that can list a pathname or pathname template for the socket?)

Connecting to existing server

1. Connect to socket.
2. Send "uid session-id\n"
   • session-id is printable ASCII, no whitespace
   • session-id limited to N (TBD) characters
     • 256 bits in base64 would be 43 characters; 512 would be 86
   • space after uid may be omitted if no session-id
3. Receive "OK\n" and wait, or "ERROR num\n" and close

Connections are not shared between threads.

Main RPC protocol: Send RPC blobs as 4-byte length then data.

Spawning new server and connecting

If initial connection attempt fails, then in a new process with the proper uid, with a pipe open to the application process, connected as file descriptor 0:

1. Detach from tty, change process group, close extraneous file descriptors, all the usual daemon stuff.
2. Create lock file (replace .sock with .lock) if it doesn't exist.
3. Acquire an exclusive advisory lock on the file.
   • Should we check for application having exited at this point?
4. Try connecting again. If it succeeds, don't send any data, just exit. Otherwise, the current process will be responsible for launching the server.
5. Remove the socket, if present.
   • This doesn't allow for the "socket" being a symlink to that in another install tree. However, one could symlink the ccapi plugin, if we make it separate from libkrb5.
6. Create the socket and listens for incoming connections.
7. Close pipe to parent.
8. Fork. Parent exits, child waits to process CCAPI connections.

Thus, the server should always be safe to launch, from a shell or from the krb5 library, without interfering with a running server process. When the subprocess closes file descriptor 0 (the pipe from the library, or just stdin from a shell) or exits, the parent should be able to make a connection attempt, and not have to worry about delaying further or looping.

In the parent (application) process

1. Create pipe and subprocess, exec server program (possibly setuid helper) in subprocess.
2. Wait for EOF on the pipe.
3. Call wait4 or waitpid on the subprocess pid (but don't use wait or wait3). Ignore errors, in case another application thread caught exit status data via a call to wait.
   • The application SIGCHLD handler may get called unexpectedly, and calls to wait functions may return unknown process ids.
   • If wait4 and waitpid aren't available, and the application isn't cleaning up after subprocesses, zombie processes may accumulate.
4. Try connection one more time.

Desired integration and release goals

Integrate and test before 1.7 release. Do not make default at this time.

Testing plan

Manual testing of all parts as work progresses.

Automated (dejagnu) testing of all parts not requiring setuid privileges.