+
|
|
|
|
+

Make Projects Fast

Updo is lightweight tooling for generating Haskell projects fast.

In the make way of only doing as much as needed, it takes seconds to generate projects with a large number of packages and dependencies. It runs locally. Alternative tools that do project mirroring can take minutes bringing down repositories of source dependencies but Updo doesn’t need to do this because it has all the needed information at hand in one configuration.

It is low-tech, made up of Makefiles, Dhall text templates and very short Haskell scripts that can optionally be installed as executables.

You’ll still need another tool to actually build the packages in these projects, tools like Cabal or Stack or haskell.nix.

Simple Bootstrapping

Updo expects one entry point, a project-files.mk (or other named) Makefile. This short file includes a recipe to bootstrap Updo, adding it to your project from hackage or its own source repository. Distributing Updo as a Haskell package makes this easy.

# project-files.mk
UPDO_VERSION ?= 1.0.0
HACKAGE := http://hackage.haskell.org/package
UPDO_URL := ${HACKAGE}/updo-${UPDO_VERSION}/updo-${UPDO_VERSION}.tar.gz

updo/Makefile:
    rm -rf updo
    curl -sSL ${UPDO_URL} | tar -xz
    mv updo-* updo
    chmod +x $$(grep -RIl '^#!' updo)

We recommend to .gitignore its updo folder for itself and its .updo working folder.

One Configuration

One set of configuration is used for all project types. You can use Updo to exclusively generate Cabal projects or exclusively generate Stack projects or do both at the same time. It can also generate a sha256map for safely grabbing packages from source repositories with haskell.nix. Different Makefile targets say what to generate. For the default .PHONY all rule in examples, we build everything for the current project but nothing else. You’re free to do whatever you want with your entry Makefile.

$ make -f project-files.mk cabal.project
$ make -f project-files.mk stack.yaml
$ make -f project-files.mk

Projects are versioned by their stackage package set resolver and matching GHC compiler version, first specified in project-versions.mk and then followed up in STACKAGE-versioned files and GHC-versioned folders for setup configuration.

# Versions of GHC and stackage resolver, the ones we're on and the next
# ones we're upgrading to.
GHC_VERSION ?= 9.2.7
STACKAGE_VERSION ?= lts-20.23

# For the upgrade, pick a matching pair of ghc-version and stack
# resolver.
GHC_UPGRADE ?= 9.4.5
STACKAGE_UPGRADE ?= lts-21.4

For the set up, we specify paths to local packages, version constraints for external published packages and source repository commits for forked or unpublished packages. Other free-form configuration in snippets can be injected into the generated projects too so all configuration options are possible. The Dhall text templates can be very simple by calling the default templates we provide or you can do whatever you want with your templates. Not all configuration is necessary. Everything except the text templates can be left out⁴.

project-dhall
└── ghc-x.y.z
    ├── constraints.dhall      ▨ List { dep : Text, ver : Text }
    ├── deps-external.dhall    ▨ List { loc : Text, tag : Text, sub : List Text }
    ├── deps-internal.dhall    ▨ List { loc : Text, tag : Text, sub : List Text }
    ├── forks-external.dhall   ▨ List { loc : Text, tag : Text, sub : List Text }
    ├── forks-internal.dhall   ▨ List { loc : Text, tag : Text, sub : List Text }
    └── text-templates
        ├── dhall2config.dhall ▨ template for cabal.project
        ├── cabalsnippet.dhall ▨ snippet  for cabal.project
        ├── dhall2stack.dhall  ▨ template for stack.yaml
        └── stacksnippet.dhall ▨ snippet  for stack.yaml

To explain the short 3-letter field names in records for:

• constraints

  { dep : Text, ver : Text }

  The version is the version equality constraint for a package dependency that can include Stack’s @rev syntax for revisions.

• source-repository-packages

  { loc : Text, tag : Text, sub : List Text }

  The location is the source repository URL, the tag is the git tag or branch name and the sub is a list of subdirectories to package .cabal files.

Copy That!

Updo is stackage-centric. If you don’t want to use stackage, this is probably not the tool for you but the dependency configuration is flexible enough to use versions other than those stackage provides in its resolver package sets.

By default, generated Cabal and Stack projects are as close to copies as we can get but why have two copies?

• Personal Preference

  Enables team members who prefer one build tool over the other to pick one.

• Get to Know Both

  Cabal and Stack have different command phraseologies. Have a project ready-to-go for trying out or learning the other tool.

• Pick the Better Tool

  Enables switching when a build feature is better in one tool or missing in the other.

• Tool Breakage Fallback

  When something breaks with one tool we have a fallback.

Progressive Upgrades

Updo is also for upgrading projects. By upgrading we usually mean upgrading to a new version of GHC and a new version of stackage but progressive upgrading can also be used for upgrading a dependency that has significantly changed or swapping one dependency for another with a different API.

Updo can work to generate one set of “current” (Cabal and Stack) projects and one set of “upgrade” projects at a time.

The “current” projects are generated by the default .PHONY all target. The active “upgrade” projects are generated by the upgrade-projects target, plural because it generates both Cabal and Stack projects. Configuration in project-dhall/ghc-x.y.z where x.y.z /= ${GHC_VERSION|GHC_UPGRADE} is ignored by Updo but “current” and active “upgrade” versions can be changed on the fly with environment variables.

$ make -f project-files.mk
$ make -f project-files.mk upgrade-projects
$ GHC_UPGRADE=9.6.3 STACKAGE_UPGRADE=nightly-2023-11-15 \
    make -f project-files.mk upgrade-projects

Local packages of the active upgrade project can be brought in progressively by keeping a project-dhall/pkgs-upgrade-todo.dhall to do list. This is best done in dependency order. The generated upgrade projects will include packages not in the to do list. That way we can compile the already upgraded packages along with packages just added, one or a few at a time for upgrade that may fail to compile. In the generated project files, the to do list is shown commented out.

Conventions

Updo is a convention over configuration tool that expects a certain minimal structure. Aside from the root level project-files.mk entry point, configuration goes into a project-dhall/ghc-x.y.z folder (it is all .dhall files). Cabal configuration from stackage will likely need to be downloaded to a project-stackage/${STACKAGE-VERSION}.config file so that we can comment out any conflicting versions in the constraints field. We can skip the download and import this configuration directly from stackage when there are no version conflicts.

Configuration of constraints (dependency package version equalities) and source repository package dependencies can be kept sorted using editor functionality because each file is a list of records. After sorting or editing we recommend using dhall format on any *.dhall file you’ve touched.

We’ve split source repository dependencies into a two-by-two matrix of original dependency or fork and internal or external package. Any of these configuration files can be omitted if not needed⁵.

$ tree
...
├── deps-external.dhall
├── deps-internal.dhall
├── forks-external.dhall
├── forks-internal.dhall

Experience Report

Updo grew to fill a need after every other way I could find to maintain projects was unsatisfactory for the specific problem I had, herding hundreds of packages and their dependencies into buildable projects over multiple compiler versions for both Cabal and Stack. The more I use it, the more I love it as a fast, simple and appropriate solution for maintaining Haskell projects.

You can find examples of Updo conversions at github/up-do , including conversions of cabal and stack.