Koji Build Interface

STATUS: draft

Abstract

This feature defines an abstract interface that Koji will use for running builds.

Rationale

Currently Koji supports a number of different build processes. Each build process is handled by a task handler (or set of task handlers) in kojid. This approach, while workable, is less than ideal for a number of reasons

  • build code is entangled with Koji code
  • build handlers must be part of Koji, or at least a plugin
  • there is a lack of consistency across build handlers

By defining this interface, we hope to:

  • make it easier to add new build processes to Koji
  • have more consistency across build processes
  • provided greater separation between the build process and Koji itself

Overview

This is an exec interface. That is, Koji will exec the underlying tool. The tool must provide subcommands and options according to specification.

Apart from the interface, Koji makes no assumptions about implementation. The buildroot tool may use any sort of container it needs to. It may run directly on the host or on a remote system.

The command line parameters for the interface are simple. Complex data is passed to the tool via files named on the command line. For example, to initialize a new build environment, the command would be:

# <kbi-tool> init <spec>

Where kbi-tool is the path to the build tool executable, and spec is a data file containing all the needed parameters for generating the build environment. The intent here is to isolate the parameters of the build in a fixed format that can be easily stored. In this case, a

Note: The build tool itself should not communicate directly with the hub api (though it may access hub content, e.g. repos, that are referenced in its parameters).

Commands

The kbi tool needs to support several different sub-commands. The name of the sub-command is given as the first argument, and the remaining arguments apply to the sub-command.

The following sub-commands are required:

init: initialize a new buildroot, based on a buildenv spec

usage: <kbi-tool> init <spec>

The single argument is a json-formatted file that specifies the parameters of the build environment.

version: the tool reports its own version to stdout

usage: <kbi-tool> version

This command takes no parameters

build: perform a build in a given environment according to spec

usage: <kbi-tool> build <buildroot_name> <spec>

The first argument is the name of the build environment to use (previously created with the init sub-command). The spec argument is a json-formatted file that specifies the parameters of the build. This is not the same as an rpm specfile.

Upon completion, the tool will provide the results of the build along with metadata about the build. The metadata will be a json-formatted file that includes the needed subset of content generator import metadata.

remove: remove the specified buildroot
usage: <kbi-tool> remove <buildroot_name>
list: list current buildroots

usage: <kbi-tool> list

This command should list all buildroots that have been created with the tool and not yet removed.

The following sub-commands are optional. Tools that opt not to implement them should return an error code of 69 (EX_UNAVAILABLE) and print an appropriate error message.

archive: create an archive of the build environment

usage: <kbi-tool> archive <buildroot_name> [<scope>]

The buildroot_name indicates the buildroot to be archived. The scope argument indicates how much of is to be archived. Valid values for scope are full and build. A full archive should include all content of the build environment. The build scope would only include the build directory from within the environment.

cleanup: clean up any stray content

usage: <kbi-tool> cleanup

If available, Koji will call this command periodically. This allows the tool to clean up any stray data that might not be removed by the remove command.

Spec files

This interface relies on a number of different spec files. Each of these will be JSON formatted. The defined spec files are:

buildenv spec: specify parameters for a build environment

The goal of this file is to codify the data needed to generate a build environment. The key data in this file will be:

  • meta: schema and version
  • buildenv: name/type/arch
  • repositories: sources to draw content from
  • build_tag: data from koji regarding the build tag
build spec: specify parameters to perform a build

For our purposes, this is a high level specification That is, this file does not deal with the sort of detail that, say, and rpm spec file does.

Key data provided:
  • meta: schema and version
  • build: build type
  • sources: list of sources to be used
  • parameters: additional parameters for the build

Build metadata

The build command should produce json metadata that is compatible with Koji’s content generator feature. Note, however, that the content generator interface is intended to import a full build across multiple architectures in one pass. This command will perform a single build in a single environment. Therefore, the metadata it produces will be one part the normal content generator metadata.

Specifically, it will include:
  • meta: schema and version
  • buildroot: metadata for the build environment used
  • output: metadata for the build output
Notable omissions relative to content generators:

  • The build heading. E.g. NVR, owner, times

    This field describes a multiarch build as a whole. Koji will collate this data.

  • The content_generator field for buildroots

    Koji will add this itself