Sign in Subscribe
irix

Bootstrap GNU Tools on SGI IRIX

Notes on trying to bootstrap GNU build tools on a very old SGI IRIX environment.

John Hartley

• 13 min read
Bootstrap GNU Tools on SGI IRIX

The objective here is pretty simple and that is to bootstrap (current) GNU build tools on SGI IRIX 6.5 machine.

SGI IRIX is a now discontinued version of UNIX which had its last release (6.5.30) in 2006.

So why bother to use SGI/IRIX rather than Linux or a Macintosh (I am not mentioning Windows here as it has no Unix heritage).

The answer will be obvious to anyone who reads my blog. I just like to fix old SGI boxes and I still use SGI/IRIX based software to do somethings that I cannot do with modern Linux machines.

Status - July 2023: Silly enough to start ...

Approach

There are lots of initiatives out there there to revive / support old SGI boxes. These include:

  • SGI Freeware - if you want to use old GNU toolsl then use can just install via SGI Freeware CD, which are readily available on the web, the problem is these are very old version of software and so not include git, which is essential if you want to build from source code nowadays
  • Nekoware Support - Nekochan was the original SGI hobbist site (now defunct) and build an binary open source software repository on top of SGI official development environment and tools. There are still people in the SGI hobbiest community, now found on "Irix Network - IRIX/Software Forum" who continue to release updates to Nekoware and there are also links to Nekoware repositories. If you like to install your open source using the SGI installation tools then this is place to go,
  • "Silicon Graphics User Group" (sgug)  RPM Software Environment (rse) - which is creating a Redhat RPM like environment for installation of the usual suspects open source software (gnu, git, openssl, python, perl etc.)
  • Other ... there is other stuff out there but much of it is ambition without resource, likely just like this blog. There is good intention but it is very sporadic and does not really provide something that is easy to use..

So what is this appoach ?

Here are the basic principles:

  • Builds software from current upstream source tree
  • Avoids private enclaves, if the software cannot be built on upstream source tree, then manage updates via open git respository and request fixes to be pushed back into upsteam respository.
  • Uses standard SGI IRIX build tools (compiler, linker etc as the SGI MIPSpro toolset provide the best MIPS code generation for IRIX ...)
  • Doesn't try to make IRIX behave like a Linux distribution (i.e. Debian app, Redhat yum/rpm)

The rationale is based the expectation is that anyone who is likely to read or replicate what I am doing here is likely be conversant with standard GNU build proces based on: configure, autoconf, automake, m4 etc..

Essentially what I am aiming for is what I used to do 20 years ago, which was have a CDROM with a bunch of GNU software tarballs on it, copy these onto the target UNIX environment and then bootstrap the GNU software needed to build the target application on that particular UNIX variant. The only difference now is that rather that have a CDROM, most of the software is retrieved via git/wget/curl etc.

As this is very much a source driven (rather than binary package driven), if i can get too the point of succesfully bootstrapping "git" then I will see if a lighter FreeBSD ports type model can be applied to old SGI IRIX platform.

The FreeBSD ports model essentially contains a repository of successfully ported software where for each port, it keeps:

  • Source Repository - to retrive code from
  • Dependencies - the dependent software that needs to be available for the code to work
  • Patch Files - the patches that need to be applied to official upstream source for it to compile on FreeBSD

This seems like a good model as it constrains the upstream software selection to version that are known to work, is source based, does not create a new enclave or private tree on the machine. To this end any software built via this process will end up in standard unix user additional tools directory space, such as: /usr/local (so binaries will be in /usr/local/bin).

NOTE: As this is a "test" I am putting stuff into "/usr/local2" for now. Then if anything major goes wrong I can just throw away this entire directory.

Measure of success:

  • small ambition - bootstrap git
  • bigger ambition - build OpenJDK on IRIX 6.5
  • along the way - try to see if can provide FreeBSD like ports respository for IRIX

Finally the desire here is not to create a grand thing with lots of effort. If the approach works then great, if it fails then lesson learnt and abandon ship.

There is no point in trying to hard on this. If this simple approach does not work then it is likely easier and recommended to just live with what is already available.

Bootstrapping Git

Git has the following defined dependencies:

  • autotools: autoconf-2.71 - depends on - m4 (m4-1.4.15 builds on IRIX without any patching, but autoconf configure complains so used m4-1.1.10)
  • curl: depends on openssl
  • zlib
  • openssl: depends on - perl, zlib
  • expat
  • libiconv

On my IRIX machine I have got SGI freeware installed. This puts all binaries in /usr/freeware/bin and includes wget, tar & gzcat

Process for GNU tools is to:

  1. wget tar.gz file (try latest release first)
  2. Expand tar.gz: gzcat XXX.tar.gz | tar xvf
  3. Change into directory
  4. Configure GNU Tool: ./configure CC=c99 CFLAGS="-64 -mips4" LDFLAGS="-64 -mips4" prefix="/usr/local2"
  5. Make tool: make
  6. If this fails then go back to version from around ~2008 and try again
  7. If older version works then split the difference between current and older and try again, try go back / forward to find most current version that works without patching on IRIX

I have recorded the "newest" version that works without patching in list above...

Need Dependency Management

As can been seen from above even the most trivial program / library ends up with dependencies that have to be managed. To help with this I have decided to see if I can "hack" FreeBSD Ports model for IRIX.

This is mostly build on top of Berkeley make (bmake), so need to bootstrap bmake.

Bootstrap bmake on IRIX

bmake has already been sliced out of BSD and is there are versions in archive going back to before 2010.

I initially bootstrap'ed bmake by:

---
--- 1. Download archive
---
# /usr/freeware/bin/wget http://www.crufty.net/ftp/pub/sjg/bmake-20161212.tar.gz
---
--- 2. Extract code
---
# gzcat bmake-20161212.tar.gz | /usr/freeware/bin/tar xvf -
---
--- 3. Configure and Build
---
# cd bmake
# ./configure CC=c99 CFLAGS="-64 -mips4 -O2" LDFLAGS="-64 -mips4" prefix=/usr/local2
# sh ./make-bootstrap.sh
---
--- 4. "make install" fails, so I just run manually...
---     (it seems to be passing invalid arguments to "install")
---
# install -c -s -u root -g sys -m 555 bmake /usr/local2/bin/
# cd mk
# install -u root -g sys *.mk /usr/local2/share/mk/
# cd sys
# mkdir /usr/local2/share/mk/sys
# install -u root -g sys *.mk /usr/local2/share/mk/sys/
# cd ../..
# install -u root -g sys *.cat1 /usr/local2/share/man/cat1/

In chasing down problem with "install make", I moved to current release. I found that this was "mostly" working, so sent in updates to upstream owner of bmake at crufty.net.

The result is a refresh of bmake IRIX support via new beta release. This has flowed into main release archive and from there likely into NetBSD and FreeBSD.  ChangeLog for updates to work with IRIX in 2023:

2023-07-15  Simon J Gerraty  <sjg@beast.crufty.net>

        * make-bootstrap.sh.in: set prefix
        If configure is run using ksh we get unexpanded ${prefix} in
        DEFAULT_SYS_PATH, by ensuring prefix is set we should still get
        correct result.

2023-07-13  Simon J Gerraty  <sjg@beast.crufty.net>

        * VERSION (_MAKE_VERSION): 20230711
        bump version for IRIX tweaks

        * make.h: undef OP_NONE if defined

        * unit-tests/Makefile: set BROKEN_TESTS for IRIX

        * configure.in: override INSTALL on IRIX

Testing with this new "beta" found some sensitivities to shell being used. This is because with IRIX 6.5 /bin/sh is actually a "Korn Shell" and not a "Bourne Shell". So to get clean build you need to use the default shell (Korn Shell) and select MIPSpro as compiler.

NOTE: The download / configuration details change based on working with upstream updates, last update Jan (strftime fix).

Updated bootstrap for bmake is:

---
--- 1a. Download (beta) archive (now use 1b.)
---    if this fails then it is because beta has flowed through main release
---
# /usr/freeware/bin/wget http://www.crufty.net/ftp/pub/sjg/beta/bmake-20230711.tar.gz
---
--- 1b. Download current release (as beta is gone ...)
---
# /usr/freeware/bin/wget http://www.crufty.net/ftp/pub/sjg/bmake.tar.gz
---
--- 2. Expand archive...
---
# gzcat bmake-XXXX.tar.gz | /usr/freeware/bin/tar xvf -
---
--- 3a. Run configure (MIPSpro 64 bit mips4 machine), put into /usr/local2
---
# cd bmake
# ./configure CC=c99 CFLAGS="-64 -mips4 -O2" LDFLAGS="-64 -mips4" prefix=/usr/local2
---
--- 3b.  Run configure to use bmake strftime rather than IRIX one (2024), due to '%s' (seconds formatting not being available on IRIX)
---
# ./configure CC=c99 CFLAGS="-64 -mips4 -O2" LDFLAGS="-64 -mips4" --with-bmake-strftime prefix=/usr/local2
---
--- 4. Build and Install
---
# sh ./make-bootstrap.sh
# make install
---
---

So this part of process went well, as rather than creating IRIX patches or having to use older release the updates needed to build with IRIX have been built into upstream bmake.

Big thank you to Simon J Gerraty  <sjg at beast.crufty.net> for helping with this.

NOTE: And please read README on beta tree for bmake at crufty.net

Getting FreeBSD Ports Tree onto IRIX

Thinking that the simplest and most source focussed way to install GNU and other Open Source software onto IRIX would be to "hack" the FreeBSD Ports model for IRIX. I need to get Ports tree on IRIX.

The simplest way to get the ports tree is via git: "git clone https://git.FreeBSD.org/ports.git /usr/ports".

So what happens if I don't have git...

My work around for this was:

  1. NFS export an area on IRIX machine in which you will do the "ports" development
  2. Make /usr/ports (or other name of your choosing) directory on IRIX machine and export this
  3. Mount the IRIX nfs exports on Linux or FreeBSD machine with current git version installed
  4. Clone ports tree to nfs mounted IRIX dev disk using Linux/FreeBSD machine
  5. On IRIX machine move ports tree to /usr/ports (or  other name of your choosing)
  6. Edit the ports tree on IRIX or Linux/FreeBSD and git push updates via Linux/FreeBSD

These NFS exported / mounted shared directories can be used until git is successfully bootstrap'ed on IRIX.

NOTE #1: Sharing a local git repository via NFS is not desirable, as it is likely to result in ownership / permissions issues. Git will complain initially and suggest remediations...

NOTE #2: To track changes to FreeBSD Ports tree, I have forked, cloned and created new "IRIX-6.5" branch.

The FreeBSD Ports tree (/usr/ports) has a directory of .mk for inclusion by Ports Makefiles. As bmake only has a single "default" built in search directory and when building ports on FreeBSD you simply go into the directory of the component you want to build and do a "make". So how does make find the <bsd.port.mk> included Makefile ? On FreeBSD this works by putting a "bsd.port.mk" in the default Make path ("/usr/share/mk"), which in turn includes the "/usr/ports/Mk/bsd.port.mk".

To replicate this for IRIX I have:

  1. Created "/usr/local2/ports/mk/irix.port.mk" which includes "/usr/ports/Mk/irix.port.mk"
  2. Added symlink from IRIX bmake install to (1): "/usr/local2/share/mk/irix.port.mk" -> "/usr/local2/share/ports/mk/irix.port/mk"

Initially I have decided to create "irix.<MACRO>.mk" versions of existing "bsd.<MACRO>/mk" files to keep IRIX files seperate from the exiting FreeBSD ones to avoid having OS specific conditions. Once the IRIX Port is working, this can be reviewed and if the number of changes in a given .mk is small, then this could be managed through OS conditional variations.

Other irix bootrap mk files include:

  • "bsd.ports.pre.mk" -> "irix.ports.pre.mk"
  • "bsd.ports.post.mk" -> "irix.ports.post.mk"

To bootstrap you need to have the "/usr/local2/share/ports/mk" .mk files, the symlinks established and the bmake build tool.

NOTE: See git repository IRIX-6.5 tree: /usr/ports/IRIX/bootstrap for bootstrapping scripts.

Bootstrap ncurses, dialog & dialog4ports

The FreeBSD ports configuration menus are create via dialog4port which in turn is dependent on the "dialog" utility program, which uses curses/ncurses.

While dialog4ports is part of the ports tree (/user/ports/ports-mgmt/dialog4ports), ncurses & dialog are part of FreeBSD OS /contrib code and is installed as part of OS.

So ncurse & dialog need to be bootstrap'ed first and then dialog4ports can be bootstrapped as the first Port to be built.

Initially I tried to build dialog4ports by linking to IRIX System V curses library, but this result in many unresolved functions, so need to build ncurces and link to this.

So ncurse & dialog has been added into bootstrap script along with bmake (see /usr/ports/IRIX/bootstrap).

Wrap FreeBSD "fetch"

FreeBSD Ports uses the FreeBSD specific "fetch" utility to get source code archives. This is built into FreeBSD OS. So to allow downloads to work on IRIX I createa a wrapper script that takes "fetch" and either invokes "/usr/freeware/bin/wget" (http only) or curl (if this is available). Initially I just try get http unconditionally and see if this is sufficient to get bootstrap working.

To get some level of https support I used old "Nekoware" curl, openssl, perl and openldap (and related libraries that need to get pulled in as depedencies). The old Nekoware seems to be sufficient to get your through the initial bootstrapping process.

Wrap FreeBSD "env"

FreeBSD Ports uses UNIX env command to setup environment (SETENV variable == "/usr/bin/env") before invoking command. The FreeBSD env takes an addition -S flag to pass in a string of environment settings which need to be expanded. So need to wrap IRIX env in script to do the -S expansion and the pass results into IRIX "/sbin/env".

Wrap FreeBSD "pkg"

FreeBSD Ports build programs from source, which are then installed using "pkg" command. The pkg support local and network repository installation and maintains database of installed programs (vs. "install" which just manages the deployment, without recording result into database).

The equivalent IRIX tool is "inst" which takes IRIX tardist (tar distribution) packages and manages there deployment, including forcing user manage dependency and conflict resolutions.

This is different to pkg which will automatically attempt to install dependencie via remote repositories. Typically inst runs interactively, and requires "go" to quick off install. There is also "-a" flag for non-interactive install.

For IRIX'ed Ports need to package target software into tardisk before it can be install, so changes to build process will be needed.

Create libbsdports Library for Missing OS functions

The FreeBDS ports management utilities (such as dialog4ports) use BSD specific functions/libraries including: err/warn API, stringlist, strsep and others.

Most of these are part of libc, so I have bundled the various missing functions (some by just bringing in FreeBSD code and other by writing some simple replacements) into "libbsdports.a". So this library needs to be bootstrapped and included in link path.

See "/usr/ports/IRIX/libbsdports" for code.

Code/Script Changes

The wrappers handle most of impact of porting to IRIX.

With the Makefile and scripts I have found the following common problems:

  • BSD Shell has: $( list )$ command substituation alternate to backquoting: ` list `- this was introduced in later verson of Korn Shell. On IRIX 6.5.30 support for this is enabled by setting environment variable _XPG=1 before shell startup (or you will need to replace $() with ``  within .mk files. See IRIX sh man page for details, in "Compatability" section.
  • The shell command execution is relying on bmake parameter substitution, rather than shell processing. Specific example of this is:
#
# example of reliance on bmake interpretaton
#
# where for loop has parameter "file"
#
do-extract: ${EXTRACT_WRKDIR}
        @for file in ${EXTRACT_ONLY}; do \
        echo "DBG>> do-extract: file='${file}' dollarsfile='$$file' workdir='${EXTRACT_WRKDIR}' cmd='${EXTRACT_CMD}' before='${EXTRACT_BEFORE_ARGS}' distfile='${_DIS
TDIR}/$$file' after='${EXTRACT_AFTER_ARGS}'."; \
        if ! ( cd ${EXTRACT_WRKDIR} && ${EXTRACT_CMD} ${EXTRACT_BEFORE_ARGS} ${_DISTDIR}/$$file ${EXTRACT_AFTER_ARGS} ); \
        then \
                ${ECHO_MSG} "===>  Failed to extract \"${_DISTDIR}/$$file\"."; \
                exit 1; \
        fi; \
        done

${_PORTS_DIRECTORIES}:
        ${MKDIR} ${.TARGET}
#
# however in code body it refers to $$file rathen then ${file}
# in fact if you use "@{file}" in this context is is empty/null
#
#
# Running above in Makefile (see "/usr/ports/IRIX/test/Makefile")
#
# bmake -d l do-extract
mkdir -p work
for file in irix-wrappers.tar; do  echo "DBG>> do-extract: file='' dollarsfile='$file' workdir='work' cmd='/usr/freeware/bin/tar' before='-xf' distfile='../$file' after=''.";  if ! ( cd work && /usr/freeware/bin/tar -xf ../$file  );  then   "===>  Failed to extract \"../$file\".";  exit 1;  fi;  done
DBG>> do-extract: file='' dollarsfile='irix-wrappers.tar' workdir='work' cmd='/usr/freeware/bin/tar' before='-xf' distfile='../irix-wrappers.tar' after=''.
#
  • It is also doing sed which is taking as input for its command generated code, so understanding the code is difficult, as you need to work out what is provided / escaped vs. what is generated. I expect if I got this wrong that I will see issues later...
  • Uses arithmetic "$(( ))" expression rather than original Korn shell "let"
  • IRIX return code - quite a few place in the scripts run file / directory removal commands ("rm -r" & "rm) where it assumes that it will continue if the file/directory does not existing. IRIX returns error code that result in make aborting. These case need to be wrapped in condition test so they continue:
#
# Example for: XX.license.mk
#
# Was:
#
        @${RM} ${_LICENSE_CATALOG_TMP} ${_LICENSE_REPORT_TMP}
#
# Change To:
#
        if [ ${RM} ${_LICENSE_CATALOG_TMP} ${_LICENSE_REPORT_TMP} ]; then \
                true; \
        else \
                echo "DBG>> license removal catalog='{${LICENSE_CATALOG_TMP}' report='${_LICENSE_REPORT_TMP}' failed continuing ..." ; \
        fi

Under certain conditions the bmake target execution will use direct shell "eval" invocation rather then forking a new shell and the Ports makefile are aware of this. Initially to ensure correct evaluation order, required breaking the "recursive" scripts into smaller parts. This is for case where make script is using "$( )" style quoting.

NOTE: In reverting scripts back fo use "$( )" style command substitution, this meant I could now use original make scripts unchanged.

Success or Failure

Do we try to go the next step (OpenJDK) or abandon ship ?

Successes:

  1. Bmake - upstream updated to support IRIX "today"
  2. GitLab & GitHub Repositories - setup public repositories, main development is via GibLab. For this and raise issues here. GitHub, only to pull in FreeBSD Ports updates (if these will ever be needed ...)
  3. Created bmake & IRIX wrappers bootstrap script (read README.IRIX)
  4. Initial testing with dialog4ports, have got fetch to work
  5. Now running from fetch -> build with crash in build config
  6. Now going up dependency tree for "fetch-recursive" target
  7. Reverted `` back quote changes to original $() syntax (appears to have not broken anything and likely will be less buggy)

Failures:

  1. See gitlab issues ;-)
  2. Initially I changed $() to `` (back quotes), but reverted back (see Success - 7.) on reading about use of _XPG=1 environment variable

References & Links:

SGI IRIX - the obligatory Wikipedia link ... once upon a time IRIX was a leading edge UNIX implementation and once of the first commercial 64 bit UNIX releases. In there day SGI was the manufactor of high end graphics workstations used with the commerical media, animation, video edition and special effects industry. Last release was in 2006 with 6.5.30, after that there where some further patches but no new minor version.

bmake - the BSD make tool, which is not maintained seperately from BSD (in in various guises).  The tarball archive has version going back to before 2010. The current release has now been updated to allow build with on IRIX and tested with MIPSpro build tools.

dialog - this is the utility that is used to build "nice" terminal forms using cursese / ncurses libraries. This dialog utility is required to build "dialog4ports", so needs to be part of basis bootstrap software

FreeBSD Ports - provides a source based mechanism to build and install open source software on FreeBSD. This is build on top of Berkeley make (bmake). So need to bootstrap bmake and also clone the FreeBSD Ports tree to get going.

IRIX'ed FreeBSD Port Tree - this is my IRIX'ed (branch) FreeBSD Port main development "gitlab repository". This is in turn a fork from my FeeBSD github respository which provides link to back to FreeBSD Port origin tree on github. So a little messy, but allow people who prefer to be a gitlab to contribute.

"IRIX Admin - Software Installation and Licensing" - official SGI IRIX documentation of inst and related tools

sgug-rse - the Silicon Graphics User Group provide the "RPM Software Environment", an RPM installation for the usual suspect open source software. I initially used this to bootstrap git, as pretty much all gnu and other common open source software is in git repositories now-a-days. However this git version had issue with github and http push, so I went to alternate git bootstrap (see "Getting FreeBSD Ports Tree onto IRIX" above)

NOTE: Image of "bootstrapper" is from - "So You Think You Can Pull Yourself Up By Your Bootstraps? (A Physics Problem)" from "Nadia In Her Own World"