Solaris Build Environment

This page contains notes on the setup of a Solaris buildbot worker for MIT krb5.

Our current Solaris build hardware is a Sun Fire V240 running Solaris 10 U10.

Solaris 10 includes a variety of free software tools in /usr/sfw, but not the full set of dependencies needed to checkout, build, and test the krb5 tree. We have chosen to use the NetBSD pkgsrc collection to build these dependencies.

Building a newer gcc

1. Set up a shell with a path containing the needed system utilities:

   /bin/bash
   export PATH=/usr/sbin:/usr/bin:/usr/sfw/bin:/usr/ccs/bin

2. Create /usr/local/src:

   mkdir /usr/local/src
   cd /usr/local/src

3. From https://ftp.gnu.org/, fetch the latest versions of gmp, mpfr, and mpc, and fetch gcc 5.4.0. Untar them in /usr/local/src.

4. Build gmp, mpfr, and mcc as follows.

   cd /usr/local/src/gmp-version
   ./configure --disable-shared ABI=32
   gmake && gmake install
   cd /usr/local/src/mpfr-version
   ./configure --disable-shared LDFLAGS=-L/usr/local/lib
   gmake && gmake install
   cd /usr/local/src/mpc-version
   ./configure --disable-shared LDFLAGS=-L/usr/local/lib
   gmake && gmake install

5. Build gcc as follows. This build takes many hours.

   cd /usr/local/src/gcc-5.4.0
   edit gcc/config/sol2.h; where it says "#ifndef USE_GLD" and subsequently talks about using a map file, change it to:
       #if !defined(USE_GLD) && defined(ENABLE_SHARED_LIBGCC)
   ./configure --with-gmp=/usr/local --with-mpfr=/usr/local --with-mpc=/usr/local --disable-shared
   gmake && gmake install

Setting up pkgsrc

If it becomes necessary to restart this process, "rm -rf /usr/pkgsrc /usr/pkg /var/db/pkg" will clean the slate.

1. Set up the shell and fetch the pkgsrc tar file:

   /bin/bash
   export PATH=/usr/pkg/sbin:/usr/pkg/bin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/sfw/bin:/usr/ccs/bin
   cd /tmp
   wget https://ftp.netbsd.org/pub/pkgsrc/stable/pkgsrc.tar.gz
   cd /usr
   gtar xzf /tmp/pkgsrc.tar.gz

2. Bootstrap pkgsrc:

   cd /usr/pkgsrc/bootstrap
   CFLAGS='-O2 -Wno-implicit-function-declaration' CC=/usr/local/bin/gcc ./bootstrap
   pkg_admin -K /var/db/pkg fetch-pkg-vulnerabilities
   (cd /usr/pkgsrc/shells/bash && bmake install)
   edit /usr/pkg/etc/mk.conf, change TOOLS_PLATFORM.sh to /usr/pkg/bin/bash
   exit out of the shell, run /usr/pkg/bin/bash, and set the path as in step 1
   (cd /usr/pkgsrc/devel/scmcvs && bmake install)
   (cd /usr/pkgsrc && cvs update)

3. Build packages useful for krb5 development:

   (cd /usr/pkgsrc/devel/git && bmake install)

Creating user accounts

By default, /home on Solaris is controlled by the automounter. To avoid needing to change the automounter configuration, we create user accounts with home directories in /export/home:

   useradd -d /export/home/ghudson -m ghudson

Creating an account allows the Kerberos principal of the same name in the ATHENA.MIT.EDU realm to log in on that account.

Problem log

A number of problems were encountered when trying simpler or mistaken variants of the above procedure. For the sake of easier debugging, here is a record of them:

• It initially looked like cvs wasn't present in pkgsrc. The directory is named devel/scmcvs, because devel/cvs would conflict with the CVS metadata directory on case-insensitive filesystems.

• When attempting to build pkgsrc with /usr/sfw/bin/gcc, a circular dependency between pkgtools/digest and lang/gcc2 was encountered. This resulted from running bmake without /usr/sfw/bin in $PATH. After correcting the path, it was necessary to run "bmake clean" in pkgtools/digest to clear the issue.

• When attempting to build pgksrc with /usr/sfw/bin/gcc, devel/p5-gettext and other packages would fail when they included <stdbool.h>, because that Solaris header is incompatible with the default standards environment of gcc 3.4.3. This can be worked around by bootstrapping pkgsrc with "gcc -std=gnu99 -Wno-implicit declaration" and setting CC to that value in /usr/pkg/etc/mk.conf after bootstrap. (Without "-Wno-implicit-declaration", the bootstrap process fails when building bsdinstall.) Building with gcc 5.4.x works around the <stdbool.h> problem, as it defaults to the gnu11 standards environment, but it also requires -Wno-implicit-function-declaration for bsdinstall to build correctly during bootstrap.

• After removing /usr/pkgsrc and /usr/pkg, bootstrapping pkgsrc again wouldn't install anything in /usr/pkg except /usr/pkg/etc/mk.conf. To start over, it is necessary to also remove /var/db/pkg.

• When attempting to build pkgsrc with "/usr/sfw/bin/gcc -std=gnu99 -Wno-implicit-declaration", /bin/bash segmentation faults were encountered in several packages (starting with p5-Error) when executing long shell lines. To work around this issue, the above procedure builds the pkgsrc bash and uses it for the build after the bootstrap.

• When attempting to build pkgsrc with "/usr/sfw/bin/gcc -std=gnu99 -Wno-implicit-declaration", pkg_create for some packages failed with stat failures, where the offending pathnames contained gcc error messages such as "gcc: unrecognized option `-print-multiarch': No such file or directory". To work around this problem, we build a newer version of gcc in /usr/local/bin before building pkgsrc.

• When building gcc's dependencies, mpc complains that libgmp uses an incompatible ABI. This is because gmp overrides the default 32-bit ABI and chooses the 64-bit ABI for performance reasons. Configuring gmp with "ABI=32" works around the issue.

• gcc does not use the configured LDFLAGS when linking some of its programs; therefore, configuring with LDFLAGS=-L/usr/local/lib is not sufficient to find its dependencies there, and configuring with LDFLAGS=-R/usr/local/lib does not allow built programs to find the shared library. To work around this problem, we configure the dependencies with --disable-shared and configure gcc with --with-gmp=/usr/local --with-mpfr=/usr/local --with-mpc=/usr/local.

• If gcc is configured with --disable-shared, the build fails with "ld: fatal: file libgcc-unwind.map: stat failed: No such file or directory". (see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65725). Building with a shared libgcc is problematic because then programs built with gcc can't be run with the standard dynamic linker path. To work around this, we apply a small change to fix the bug.