The Fantom compiler is written in Fantom itself - which presents a chicken and the egg problem. How do you compile the compiler without having a compiler? The problem is further compounded by the fact that the build scripts themselves are written in Fantom. To solve this problem, the bootstrap process requires two Fantom installations:
- rel: known good Fantom installation (typically the last build)
- dev: development environment to build (probably pulled from hg)
By convention we structure our development directory tree like this:
dev/ rel/ bin/ lib/ ... fan/ bin/ lib/ src/ ...
The "rel" directory always contains the last released build. The "fan" directory contains our main development code branch. We call our top level directory "dev", but for the purposes of this discussion "fan" is the development directory.
The easiest way to perform a bootstrap build from hg tip is to use the "bootstrap.fan" script:
- Download latest released build from http://fantom.org/
- Unzip that build to
hgis installed in your path (see Mercurial)
java_homeenv var points to your JDK installation
- Go grab a cup of coffee
NOTE: you need JDK 1.6 to recompile Fantom. However only 1.5 is required to run Fantom.
By default devHome is assumed to be a peer to relHome called "fan". If you'd like to use another directory then specify the "-devHome" option.
This script will perform a number of steps:
- Verifies your environment
- Clones hg tip or if hg repo exists performs a pull/update
- Configures your etc files
- Rebuilds your devHome environment from scratch
NOTE: the bootstrap script does not update your substitutes. You must still do that by hand if you plan on rebuilding individual bootstrap pods yourself.
The rest of this chapter covers all the internal details required to fully understand bootstrap building.
By default the build assumes devHome to be the home directory of Fantom installation. For example if you rebuild jfan, then the output goes into "devHome/lib/java/sys.jar". In the case of the rel installation we don't want this default because we will overwrite ourselves (which leads to some nasty problems). So you need to set the devHome prop in "etc/build/config.props" of your rel installation to reference the dev directory using a URI (not OS path):
// Must be configured boot build in substitute/release installation devHome=file:/C:/dev/fan/
If you forget to do this, then you will get a build error which looks something like this:
C:\dev\fan\src>sys\java\build ERR: Must update 'devHome' for bootstrap build BUILD FAILED [12ms]!
On a clean machine with only source code, we don't have any pods compiled such as
compiler. In order to run the build scripts to compile these pods, we need to use our rel installation.
To make this all work seamlessly on Windows, the Fantom launcher will look in "etc/sys/config.props" to see if a substitute runtime should be used. For example in our environment we map the following scripts to use the rel installation.
// etc/sys/config.props of your *dev* directory runtime.substitutes= \ /C:/dev/fan/src/buildall.fan = C:\\dev\\rel \ /C:/dev/fan/src/buildboot.fan = C:\\dev\\rel \ /C:/dev/fan/src/jfan/build.fan = C:\\dev\\rel \ /C:/dev/fan/src/sys/build.fan = C:\\dev\\rel \ /C:/dev/fan/src/compiler/build.fan = C:\\dev\\rel \ /C:/dev/fan/src/build/build.fan = C:\\dev\\rel
The launcher will check if any script being run matches one of those files. If a match is made, then it will route to the alternate runtime specified. Turn on launcher debugging to see exactly what is happening under the covers.
Note that the extension used to run the build scripts needs to match the substitutes defined in the "config.props". If you set your
pathext to include ".fan", then Windows will pass the extension into the launcher. If you are running an alternate way you might need to explicitly specify the extension on your command line or change the extensions defined in "config.props".
Unix substitution is implemented by the bootstrap build scripts using the following shebang:
#! /usr/bin/env fansubstitute
The fansubstitute script explictly sets FAN_HOME to the value of the FAN_SUBSTITUTE variable before launching. So you will need to export FAN_SUBSTITUTE to reference your rel installation. And of course you have to run your scripts as executables so that the shebang takes effect.
Also note that building on Unix will skip any .NET targets.
JDK and .NET Tools
You need JDK 1.6 or greater to compile from source (only 1.5 is required for the Java runtime).
Version 2.0 or greater of .NET is required.
In order to compile from source you will need to setup some config properties to reference your JDK and .NET home directories:
These paths should be formatted as file system Uris (not OS paths). These properties should be configured in "etc/build/config.props" of both your rel and dev environments.
The .NET targets are automatically skipped if not running Windows. So you can ignore the .NET configuration on Unix platforms.
The "buildall.fan" script is the top level build script for compiling the Fantom distribution. We commonly run this command to rebuild everything and run tests on every pod:
buildall full test
The "buildall" script is executed by the rel substitute runtime and in turn launches two sub-scripts. The "buildboot.fan" script manages rebuilding the core runtime modules:
sys/build.fan sys/java/build.fan sys/dotnet/build.fan sys/js/build.fan compiler/build.fan compilerJava/build.fan build/build.fan
Once the bootstrap modules are compiled, the development environment is self hosting and can be used to compile the remainder of itself. This is done via the "buildpods.fan" script.
The bootstrap issue can cause some confusing dependency issues which are summarized here:
- The rel compiler will be generating the
buildpod files. This means that the rel compiler must be able to generate fcode that the dev runtime can read. It also means that the rel compiler must be able to read any new syntax used by dev versions of
- The rel compiler will actually use the dev versions of the pods to resolve dependencies. For example dev compiler can reference new sys APIs defined in dev but not rel. Under the covers this works because the
buildpod's build scripts specify a non-default
Because of these restrictions, adding new language features and fcode changes require some careful planning.
You can use the "dumpEnv" build target to dump key aspects of your build script environment. You can run target on "buildall.fan", although a more concise report is to dumpEnv on "buildboot.fan" and one of the non bootstrap pods like "testSys/build.fan".
Key things to note about your environment:
- all scripts should be using dev for devHomeDir
- bootbuild scripts should be using rel for fanHome
- non-bootbuild scripts like testSys should be using dev for fanHome
- verify javaHome for sys/javafan/build.fan
- verify dotnetHome for sys/dotnet/build.fan (if running on Windows)
In summary, you want to make sure of a couple key things:
- setup your rel installation and never touch it (consider it readonly)
- ensure jdkHome and dotnetHome are configured in both rel and dev etc/build/config.props
- ensure rel etc/build/config.props devHome points to your dev installation
- make sure your substitutes are configured correctly:
- on Unix make sure your FAN_SUBSTITUTE env points to the rel installation
- on Windows make sure your dev etc/sys/config.props substitutes are configured
- only put dev bin your path and always run your scripts from the dev installation
- never use a working repo for bootstrap (use only the boot repo)