Projects/Plugin support improvements (first version)

Motivation

We set up our goal on creating a framework that

• Separates plugin interface from its implementation;
• Provides simple and clear mechanism that facilitates additions of new plugin interfaces and their implementations(modules);
• Handles both built-in and dynamic plugin modules;
• Allows multiple implementation of the same plugin interface;
• Provides uniform way to supply parameters for plugin configuration;
• Allows one plugin implementation to use services provided by the other plugin implementations.

Structure

Participants

• Plugin Manager (PM) - Module that implements operations that manage plugin configuration and registry services. PM is responsible to maintain a container populated with plugin handles (references to plugin objects) and provide the search capabilities for callers requesting the plugin services.
• Plugin Loader (PL) - Abstract module that declares an interface for operations that facilitate manipulation with the set of plugin implementations;
• Plugin Loader Implementation (PLI) - Concrete implementation of PL. It knows about the set of the available implementations and how to create them;
• Plugin Interface (PI) - Abstract module that declares plugin interface.
• Plugin Interface Implementation (PII) - Concrete implementation of the plugin interface;
• Caller - Plugin caller.

Collaboration

• An instance pl_manager of PM is created as part of krb5_init_context operation at run-time (1). This instance will use PM methods to configure and register the desired plugin implementations;
• The list of the desired plugin implementations is known to the specific PLI which is aggregated within PM (2);
• PM delegates acquiring of plugin interfaces to PLI (3);
• The caller uses pl_manager to invoke the desired plugin interface (4).

User Interface

plugin_manager_get_service

- perform a search in the plugin manager registry to locate the particular interface based on plugin interface name and plugin id. If the search is successful, the plhandle structure gets populated with reference to the plugin interface and the function return code is set to success. Otherwise, this function returns an error;

plugin_manager_configure

- parse a configuration file. Based on the obtained information construct plugin objects and load them into the plugin registry;

plugin_manager_start

- complete necessary initializations of plugin objects and make them available for use;

plugin_manager_stop

- free resources;

plugin_manager_get_instance

- instantiate an instance of plugin manager if it does not exist.

plugin_loader_create_api

- return the vtable with the implementation pertained to the specific interface;

Implementation

MT safety 

Implementation must be MT safe.

Proof of concept code is available from plugins branch

svn+ssh://svn.mit.edu/krb5/branches/plugins

k5-int.h

plugin_manager *pl_manager is added to _krb5_context;

PM

The purpose of PM is to provide the configuration and registry services. In future the ability to have multiple implementations of the PM may be desired to allow one to have flexible configuration environment (For example, the configuration service can be implemented using various file formats or be completely hardcoded.)

PM vs PLI

In general PM is tied to one or more PLIs. In its turn, each PLI is responsible for loading one or more plugins. The notion of the multiple plugins per PLI is justified by the convenience of the grouping based on the plugin functionality, level of optimization or other features;

Consequences

The framework is designed to hide the management details, such as configuration and loading, from the plugin writers and callers.

• The responsibility of the writer of the new plugin interface (PI) is limited to the writing source and header as described at Template_1:

Template 1

HEADER
#include <plugin_manager.h>
#include <k5-int.h>
typedef struct {
               int version;
               char plugin_name[MAX_PL_NAME_LEN];
               krb5_error_code (*fn1)(...);
} plugin_PLName;
krb5_error_code plugin_PLName_fn1(plhandle handle, ...); 
 
SOURCE
krb5_error_code plugin_PLName_fn1(plhandle handle, ...)
{
   plugin_PLName* api = (plugin_PLName*) handle.api;
   api->plugin_PLName_fn1(...);
   return 0;
}

• The responsibility of the writer of the new plugin interface implementation (PII) is limited to the writing source and header as described at Template_2 and the notifying the plugin loader about its availability Template_3:

Template 2

HEADER
plhandle plugin_pwd_qlty_krb_create(void);

SOURCE
static krb5_error_code
plugin_PLName_fn1(...)
{ /*intended functionality */}
 
static void
plugin_PLName_cleanup(...) 
{ /* cleanup */ }

static krb5_error_code
plugin_PLName_init(...)
{ /* init */ } 

plhandle  plugin_PLName_PLImpl_create()
{
   plhandle handle;;
   plugin_PLName* api = malloc(sizeof(plugin_PLName));
   api->fn1 = plugin_PLName_fn1;
   api->PLName_init = plugin_PLName_init;
   api->PLName_cleanup = plugin_PLName_cleanup;
   handle.api = api;
   return handle;
}

Template 3

static plugin_descr _table[] = {
   {"plugin_PLImpl_PLName", plugin_PLName_PLImpl_create},
};

For better readability some of the verification, clean-up, error handling code is omitted from the templates

Built-in Plugins

• Examples

  PreAuth, AuthData, Password Quality (see Current plugins section for details)

• Invoking multiple implementations of the same plugin

  This is achieved by using the aimed plugin_id's in plugin_manager_get_service calls. For example, two specific implementations of the password quality plugins may be invoked in password check.

• Invoking all available plugin implementations

  See also "Defaults" section

Dynamic Plugins

• Examples

  Audit, Password sync

• Requirements

1. The difference between dynamic and built-in plugins (both interface and implementation) should be on the level of linkage.
2. Plugin consumer should not have the knowledge about the type of plugins it is using (in terms of dynamic/built-in). On the caller side the code invoking plugin should stay the same for both dynamic and built-in plugins.
3. Similar to built-in plugins, the plugin consumer may call the multiple implementations of the same plugin interface. For example, two implementations of the audit plugin interface may be used to handle password change and intruder lockout events.
4. Similar to built-in plugins, one can choose to use all available plugin implementations. Their location may be some default one, or passed via env variable, or defined in the configuration.
5. In the case one wants to have better control over the configuration and management of the plugins selection, it responsibility of the administrator to point to the location of the dynamic plugin as part of the configuration.

• Creating dynamic plugin

  The appropriate loader is linked with the set of the desired plugins resulting into the shared library. Details:

1. One of the deliverables of the plugin framework is the loader file. Its responsibility is to load modules that are listed in its local table (see below). ;
2. The dynamic plugin creator must fill this table with the list of the desired pluginmodules.
3. The dynamic plugin creator then links the loader with the set of the desired plugin modules resulting into the shared library;
4. Administrator points to the location of the dynamic plugin by setting plugin_loader_path configuration attribute to this library.

NOTE: This approach allows to bundle dynamic plugins. 
      It also hides loading details from the plugin writer making code for dynamic and built-in plugin indistinguishable. 

The following is the example of how the table in the loader may look:

 static plugin_descr  plugin_dyn_factory_table[] = {
   {"plugin_pwd_qlty_DYN",   plugin_pwd_qlty_DYN_create},
   {"plugin_pwd_qlty_XYZ",   plugin_pwd_qlty_XYZ_create},
   {NULL,NULL}
};

• Invoking dynamic plugin

  Similar to builtin plugins with the following additions. The option plugin_loader_path in the configuration file refers to the location of the dynamic plugin shared library and the option plugin_loader_type must be set to dynamic.

Configuration

If no plugin configuration is provided the new plugin framework should act upon some default behavior.

• Configuration file attributes

plugin list - list of the desired plugins;

plugin_api - name of the plugin interface;

plugin_name - concrete plugin implementation;

plugin_version - desired version of plugin implementation; This may be used by the administrator to point to the most recent plugin implementation (perhaps, bug fix).

plugin_loader_name - name of the loader the particular implementation is associated with;

plugin_loader_type - defines the plugin type: built-in or dynamic;

plugin_loader_path - path to the plugin dynamic library

plugin_disable - don't use this plugin implementation

plugin_properties - plugin implementation initialization parameters (when needed)

• Example of the plugin section in krb5.conf

[plugins]
       /* By default, all built-in plugin impls are enabled. To disable a particular built-in plugin one need to specify it */
       plugin_list = PQ1
       PQ1 = {
               plugin_api = plugin_pwd_qlty        
               plugin_name = plugin_pwd_qlty_krb   
               plugin_disable = yes
       }
      
       /* dynamic plugin */
       plugin_list = PQ_DYN
       PQ_DYN = {
              plugin_api = plugin_pwd_qlty
              plugin_name = plugin_pwd_qlty_DYN
              plugin_loader_type = dynamic
              plugin_loader_path = /path-to-the-lib/libplugin_dynamic.so   # if default, may be omitted
       }

• Alternatives

The choice of plugin configuration format is a matter of preference. It may be presented in XML, YAML, JSON etc. One needs only an appropriate parser.

• Configuration change at run-time

TBD

Directory Structure

• src/plugin_core

  plugin_manager.[ch], plugin_loader.[ch], impl/plugin_XXX_manager.[ch], impl/plugin_XXX_loader.[ch]

• src/plugins

  pluginA/plugin_A.[ch], pluginA/plugin_A_implN/plugin_A_implN.[ch]

• src/plugin_dynamic

  plugin_dyn_factory.[ch]

Defaults

If no plugin configuration is provided new plugin framework should act upon some default behavior.

There are two default loaders. The first loader aggregates built-in plugin implementations, while the second one - dynamically loadable ones located in some default directory. If the administrator wishes to have all of these plugin implementations available and they need no additional configuration (for example, for their initialization), the configuration file may not have any plugin related information.

Build process

• Options

  DEBUG_PLUGINS - output more info for debugging

Sample Code

• Plugin initialization

The following is an example of the plugin manager initialization:

plugin_manager_get_instance(&ctx->pl_manager);
plugin_manager_configure(ctx->pl_manager, conf_path);
plugin_manager_start(ctx->pl_manager);

• How to call plugins

The following is an example of invoking Password Quality plugin:

plugin_manager_get_service(ctx->pl_manager, PWD_QLTY, PWD_QLTY_KRB, &plugin_handle);
plugin_pwd_qlty_check(plugin_handle, srv_handle, password, use_policy, pol, principal);

• How to construct plugin interface (PI)

The following is an example of how plugin interface called "password quality" may look:

#ifndef PLUGIN_PWD_QLTY_H_
#define PLUGIN_PWD_QLTY_H_

typedef struct {
       int version;
       char plugin_id[MAX_PL_NAME_LEN];
       kadm5_ret_t (*pwd_qlty_init)(kadm5_server_handle_t);
       void (*pwd_qlty_cleanup)();
       kadm5_ret_t (*pwd_qlty_check)(kadm5_server_handle_t, char*,  int, kadm5_policy_ent_t, krb5_principal);
} plugin_pwd_qlty;

kadm5_ret_t plugin_pwd_qlty_init(plhandle, kadm5_server_handle_t);
kadm5_ret_t plugin_pwd_qlty_cleanup(plhandle);
kadm5_ret_t plugin_pwd_qlty_check(plhandle, kadm5_server_handle_t, char*, int,  kadm5_policy_ent_t, krb5_principal);

#endif /* PLUGIN_PWD_QLTY_H_ */
 

SOURCE 
kadm5_ret_t 
plugin_pwd_qlty_check( plhandle handle, kadm5_server_handle_t srv_handle, char *password, int use_policy, kadm5_policy_ent_t pol, krb5_principal principal) 
{
   plugin_pwd_qlty* api = (plugin_pwd_qlty*) handle.api;
   api->pwd_qlty_check(srv_handle, password, use_policy, pol, principal);
}

kadm5_ret_t
plugin_pwd_qlty_init(plhandle, kadm5_server_handle_t)
{ /*  init */ } 

void
plugin_pwd_qlty_cleanup(plhandle) 
{ /* clean-up */ } 

• How to construct plugin interface implementation (PII)

Every PII should have only one public create function. For PII called "password quality krb" it would be plhandle plugin_pwd_qlty_krb_create().The following is an example of the this PII:

#ifndef PLUGIN_PWD_QLTY_KRB_IMPL_H_
#define PLUGIN_PWD_QLTY_KRB_IMPL_H_

plhandle plugin_pwd_qlty_krb_create(void);

#endif /* PLUGIN_PWD_QLTY_KRB_IMPL_H_ */
 

SOURCE
plhandle plugin_pwd_qlty_krb_create()
{
   plhandle handle;
   ...
   api->pwd_qlty_init    = plugin_pwd_qlty_init;
   api->pwd_qlty_check   = plugin_pwd_qlty_check;
   api->pwd_qlty_cleanup = plugin_pwd_qlty_clean;
   handle.api = api;
   return handle;
}
 
static kadm5_ret_t 
plugin_pwd_qlty_check(kadm5_server_handle_t srv_handle, char *password, int use_policy, kadm5_policy_ent_t pol, krb5_principal principal)
{ /* do some quality verification */ }
 
static  kadm5_ret_t  
plugin_pwd_qlty_init(kadm5_server_handle_t handle)
{ /* initialization */ } 
 
static kadm5_ret_t   
plugin_pwd_qlty_cleanup()
{ /* cleanup */ } 

For better readability some of the verification, clean-up, error handling code is omitted from the samples

Current plugins

We currently have the following plugin:

• Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

• Authdata: Very similar to the preauth framework.

• KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

• KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

• GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

• Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

• ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

• keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

• Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

Future plugins

The following areas are candidates for future plugin support:

• PRNG
• profile / configuration
• DNS / host-realm mapping
• password quality policy
• lockout
• audit
• password synchronization

Current support infrastructure

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

• krb5int_open_plugin - Create a plugin handle from a filename
• krb5int_close_plugin - Close a plugin handle
• krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
• krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

• krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
• krb5int_close_plugin_dirs - Close a plugin dir handle
• krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
• krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
• krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
• krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

Problem areas

• Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

• Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

• In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

• We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

• When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

• Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

• In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

Definitions

pluggable interface

an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

module

a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.

built-in

a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (Dependent library shared objects of the calling library can contain "built-in" modules for the calling library.) The distinguishing feature of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.

dynamically loaded

a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space.

discovery

process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.

• compiled-in
• directory scan
• explicit inclusion by configuration
• explicit exclusion by configuration

loading

the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.

• built-in
• dynamic loading
• application-registered

selection

the process of a caller invoking one specific module from the set of loaded modules that implement an interface.