CMake (no dependencies)

CMake project without third party dependencies can be used as is in Hunter. However you need to check that CMake code is correctly written and use best CMake practices.

Default behavior

Please check that your package respect (i.e. does not rewrite) such CMake variables like:

Environment

Configuration of the package should be predictable.

For example it should not depend on the fact that some package already installed or not:

find_package(OpenSSL)
if(OPENSSL_FOUND)
  target_compile_definitions(... PUBLIC FOO_WITH_OPENSSL=1)
endif()

If package is optional then control behavior explicitly:

option(FOO_WITH_OPENSSL "Build with OpenSSL" ON)

if(FOO_WITH_OPENSSL)
  find_package(OpenSSL REQUIRED) # fatal error if not found!
  target_compile_definitions(... PUBLIC FOO_WITH_OPENSSL=1)
endif()

Same with the programs:

find_program(PYTHON_EXE python) # Use 'find_package(PythonInterp)' in real code
if(PYTHON_EXE)
  # generate some extra code
endif()

Use this code instead:

option(FOO_WITH_PYTHON "Build with Python" ON)

if(FOO_WITH_PYTHON)
  find_program(PYTHON_EXE python)
  if(NOT PYTHON_EXE)
    message(FATAL_ERROR "Python not found")
  endif()
endif()

Environment variable example:

if(EXISTS "$ENV{FOO_EXTRA_CODE}")
  # add some code
endif()

Solution:

option(FOO_WITH_EXTRA_CODE "Use extra code" ON)

if(FOO_WITH_EXTRA_CODE)
  if(NOT EXISTS "$ENV{FOO_EXTRA_CODE}")
    message(FATAL_ERROR "...")
  endif()
endif()

Note that this is kind of a natural limitation because otherwise Hunter have to save the whole outside environment like default paths, environment variables, etc. This is not doable on practice.

Exception is the variables related to compiler/toolchain like compiler version, compiler id, platforms, generators, architectures: WIN32, IOS, ANDROID, etc. Number of such traits is limited and forms toolchain-id.

CGold

Install XXXConfig.cmake

The easiest way to integrate installed libraries into other project is to use find_package command. Project should generate and install *Config.cmake files instead of using Find*.cmake modules. It’s the one of the painless ways to support relocation - imported targets can be cached and downloaded as prebuilt binary archive from build servers. Plus only imported targets works nicely with non standard build types like MinSizeRel or RelWithDebInfo.

To check this feature you can try to install files to local directory. If result of installation looks like this:

Install the project...
/.../cmake -P cmake_install.cmake
-- Install configuration: "Release"
-- Installing: /.../lib/libhunter_box_1.a
-- Installing: /.../include/hunter_box_1.hpp

It means that this feature is missing and you need to patch CMake code to introduce it. Details can be found here.

Installation after fix:

Install the project...
/.../cmake -P cmake_install.cmake
-- Install configuration: "Release"
-- Installing: /.../lib/libhunter_box_1.a
-- Installing: /.../include/hunter_box_1.hpp
-- Installing: /.../lib/cmake/hunter_box_1/hunter_box_1Config.cmake
-- Installing: /.../lib/cmake/hunter_box_1/hunter_box_1ConfigVersion.cmake
-- Installing: /.../lib/cmake/hunter_box_1/hunter_box_1Targets.cmake
-- Installing: /.../lib/cmake/hunter_box_1/hunter_box_1Targets-release.cmake

CGold

CMake documentation

Add package to Hunter

Next let’s assume user hunterbox is trying to add hunter_box_1 project to Hunter.

Examples on GitHub

Recommended name for the package is lowercase separated with underscore.

C++:

#include <hunter_box_1.hpp>

int main() {
  hunter_box_1::foo();
}

// file hunter_box_1.hpp

namespace hunter_box_1 {
} // namespace hunter_box_1

CMake with Hunter:

hunter_add_package(hunter_box_1)
find_package(hunter_box_1 CONFIG REQUIRED)
target_link_libraries(... hunter_box_1::hunter_box_1)

In Hunter sources:

  • cmake/projects/hunter_box_1/hunter.cmake file with versions
  • examples/hunter_box_1 directory with example for testing
  • docs/packages/pkg/hunter_box_1.rst documentation for package

Git branches (will be covered in details later):

  • pkg.hunter_box_1 branch for testing
  • upload.hunter_box_1 branch for uploading
  • pr.hunter_box_1 work-in-progress branch for adding/updating package

Fork Hunter

Hunter hosted on GitHub service where common way to add code is to fork project and create pull request.

Fork ruslo/hunter, clone your fork and initialize all submodules:

> git clone https://github.com/hunterbox/hunter
> cd hunter
[hunter]> git submodule update --init --recursive .

Create branch to work on new package:

[hunter]> git checkout -b pr.hunter_box_1

Add versions

Add one or several versions of hunter_box_1 package to corresponding hunter.cmake file.

Copy template and substitute all strings foo to hunter_box_1:

[hunter]> cp -r cmake/projects/foo cmake/projects/hunter_box_1
[hunter]> sed -i 's,foo,hunter_box_1,g' cmake/projects/hunter_box_1/hunter.cmake

Download release archive and calculate SHA1:

> wget https://github.com/hunterbox/hunter_box_1/archive/v1.0.0.tar.gz
> openssl sha1 v1.0.0.tar.gz
SHA1(v1.0.0.tar.gz)= c724e0f8a4ebc95cf7ba628b89b998b3b3c2697d

Add this information to cmake/projects/hunter_box_1/hunter.cmake file:

# !!! DO NOT PLACE HEADER GUARDS HERE !!!

include(hunter_add_version)
include(hunter_cacheable)
include(hunter_download)
include(hunter_pick_scheme)

hunter_add_version(
    PACKAGE_NAME
    hunter_box_1
    VERSION
    1.0.0
    URL
    "https://github.com/hunterbox/hunter_box_1/archive/v1.0.0.tar.gz"
    SHA1
    c724e0f8a4ebc95cf7ba628b89b998b3b3c2697d
)

hunter_pick_scheme(DEFAULT url_sha1_cmake)
hunter_cacheable(hunter_box_1)
hunter_download(PACKAGE_NAME hunter_box_1)

CMake options

Note that it does not make sense to include in build such stuff like examples, tests or documentation. Please check that your package has CMake option to disable it and apply extra variables to all versions (if options is not disabled by default) using hunter_cmake_args function:

# bottom of cmake/projects/Foo/hunter.cmake

hunter_cmake_args(
    Foo
    CMAKE_ARGS
        FOO_BUILD_EXAMPLES=OFF
        FOO_BUILD_TESTS=OFF
        FOO_BUILD_DOCUMENTATION=OFF
)

hunter_pick_scheme(DEFAULT url_sha1_cmake)
hunter_download(PACKAGE_NAME Foo)

Options set by hunter_cmake_args have lower precedence than options set by hunter_config(... CMAKE_ARGS ...) (see order).

Build types

Warning

Usually there is no need to set build type explicitly. If package does not work with default Debug + Release it means something wrong with package itself.

Default build type(s) can be set by hunter_configuration_types:

hunter_configuration_types(Foo CONFIGURATION_TYPES Release)
hunter_download(PACKAGE_NAME Foo)

User can overwrite this default by using custom hunter_config parameters.

Set default version

Add hunter_config directive with default version to cmake/configs/default.cmake:

hunter_config(hunter_box_1 VERSION 1.0.0)

Create example

Simple example will be used to test integration of package. Copy template example and substitute all strings foo with hunter_box_1:

[hunter]> cp -r examples/foo examples/hunter_box_1
[hunter]> sed -i 's,foo,hunter_box_1,g' examples/hunter_box_1/*

Tweak all files in examples/hunter_box_1 directory to fit headers and name of imported targets.

Add documentation

Each package should have page with information and usage example.

To create such page copy template file and substitute all strings foo with project name hunter_box_1 string:

[hunter]> cp docs/packages/pkg/foo.rst docs/packages/pkg/hunter_box_1.rst
[hunter]> sed -i 's,foo,hunter_box_1,g' docs/packages/pkg/hunter_box_1.rst

Open file docs/packages/pkg/hunter_box_1.rst and tweak all entries.

Substitute unsorted with some tag in directive .. index:: unsorted ; foo. This tag will be used on this page.

See also

Note

Since you don’t know a priory pull request number leave it as N for now. You can update it later.

To check documentation building locally you can run:

[hunter]> cd docs
[hunter/docs]> source ./jenkins.sh
(_venv) [hunter/docs]> ./make.sh

Commit

Now save all changes by doing commit:

[hunter]> git branch
  master
* pr.hunter_box_1

[hunter]> git add cmake/configs/default.cmake
[hunter]> git add cmake/projects/hunter_box_1/
[hunter]> git add docs/packages/pkg/hunter_box_1.rst
[hunter]> git add examples/hunter_box_1/

[hunter]> git commit -m "Add 'hunter_box_1' package"

Testing locally

This step is optional since we will run tests on CI server. However it’s the fastest way to check that everything is ready and working correctly.

Script jenkins.py will package temporary Hunter archive based on current state and build example. This script use Polly toolchains.

Check you have Python 3 installed, clone Polly and add bin folder to PATH environment variable:

> which python3
/usr/bin/python3

> git clone https://github.com/ruslo/polly
> cd polly
[polly]> export PATH="`pwd`/bin:$PATH"

Go back to Hunter repository and run test:

> cd hunter
[hunter]> which polly.py
/.../bin/polly.py
[hunter]> TOOLCHAIN=gcc PROJECT_DIR=examples/hunter_box_1 ./jenkins.py

CI testing

Now let’s run tests on continuous integration servers with various toolchains and platforms. Hunter use AppVeyor to test Windows (Visual Studio, NMake, Ninja, MinGW, MSYS) and Travis to test Linux (GCC, Clang, Android, Clang Analyzer, Sanitize Address, Sanitize Leak) and OSX (Clang + Makefile, Xcode, iOS).

Register your Hunter fork:

Branch master

To understand what should be done next you need to understand the structure of branches.

In branch master there is only .travis.yml file which will only check documentation building:

When you open pull request to ruslo/hunter this test will run automatically.

Branch pkg.template

In branch pkg.template of ingenue/hunter repository there are template .travis.yml and appveyor.yml files:

All changes from master will go to pkg.template. The only difference between master and pkg.template is .travis.yml/appveyor.yml files.

Branch pkg.<name>

Branch for testing package <name>.

Real testing happens in pkg.<name> branch of ingenue/hunter repository. E.g. branch pkg.gtest:

All changes from pkg.template will go to pkg.<name> branch on updates. The only difference between pkg.template and pkg.<name> is travis.yml/appveyor.yml files.

Branch upload.<name>

Branch for uploads.

After successful tests on pkg.<name> branch upload.<name> will do uploads. E.g. branch upload.gtest:

All changes from pkg.<name> will go to upload.<name> branch on updates. The only difference between upload.<name> and pkg.<name> is build command: jenkins.py vs. jenkins.py --upload.

Branches structure

Here is an image for branches structure:

Branches

Fetch CI configs

Since we are adding new package we have to create new CI configs for it. Fetch pkg.template branch and substitute all foo strings with hunter_box_1:

[hunter]> git remote add ci https://github.com/ingenue/hunter
[hunter]> git fetch ci
[hunter]> git checkout pkg.template
[hunter]> git checkout -b pr.pkg.hunter_box_1

[hunter]> sed -i 's,foo,hunter_box_1,g' .travis.yml
[hunter]> sed -i 's,foo,hunter_box_1,g' appveyor.yml

[hunter]> git add .travis.yml appveyor.yml
[hunter]> git commit -m "Test 'hunter_box_1' package"

Run remote tests

Current state is:

  • pr.hunter_box_1 contains new package
  • pr.pkg.hunter_box_1 contains configs for testing
[hunter]> git branch -vv
  master
  pkg.template
  pr.hunter_box_1     9f60453 Add 'hunter_box_1' package
* pr.pkg.hunter_box_1 4a7626d Test 'hunter_box_1' package

To run tests we need to merge both changes into test.hunter_box_1 and push test.hunter_box_1 branch to remote:

[hunter]> git checkout pr.hunter_box_1
[hunter]> git checkout -b test.hunter_box_1
[hunter]> git merge pr.pkg.hunter_box_1
[hunter]> git push -u origin test.hunter_box_1

Example:

Pull request

Excluding tests

If all tests passed you can skip this section.

If some toolchains are working and some toolchains failed it means project has platform-specific problems. Note that you don’t have to have all toolchains working and there is no need to fix all issues you see. If at least something is working then you can exclude broken toolchain and you or somebody else can apply fixes later.

Do not remove toolchains from .travis.yml/appveyor.yml configs. Comment it out instead to simplify enabling it back. Do not change the order of toolchains since it will affect git merge. Leave the link to broken job:

--- /home/docs/checkouts/readthedocs.org/user_builds/pol51-hunter/checkouts/stable/docs/creating-new/create/ci/.travis-OLD.yml
+++ /home/docs/checkouts/readthedocs.org/user_builds/pol51-hunter/checkouts/stable/docs/creating-new/create/ci/.travis-NEW.yml
@@ -54,8 +54,12 @@
       env: PROJECT_DIR=examples/hunter_box_1 TOOLCHAIN=libcxx
     - os: osx
       env: PROJECT_DIR=examples/hunter_box_1 TOOLCHAIN=osx-10-11
-    - os: osx
-      env: PROJECT_DIR=examples/hunter_box_1 TOOLCHAIN=ios-nocodesign-9-3
+
+    # FIXME: iOS is not supported by this package
+    # * https://travis-ci.org/hunterbox/hunter/jobs/276514722
+    # - os: osx
+    #   env: PROJECT_DIR=examples/hunter_box_1 TOOLCHAIN=ios-nocodesign-9-3
+
     # }
 
 install:

If no working toolchains left for .travis.yml or appveyor.yml then comment out everything and add TOOLCHAIN=dummy test.

Go to branch pr.pkg.hunter_box_1 with CI configs and commit this change there:

[hunter]> git checkout pr.pkg.hunter_box_1
[hunter]> git add .travis.yml
[hunter]> git commit -m 'Exclude broken'

Go to testing branch test.hunter_box_1, merge updated CI configs and run new CI tests by pushing commits to remote:

[hunter]> git checkout test.hunter_box_1
[hunter]> git merge pr.pkg.hunter_box_1
[hunter]> git push

Pull requests (tests)

First push pr.pkg.hunter_box_1 with CI configs:

[hunter]> git checkout pr.pkg.hunter_box_1
[hunter]> git push -u origin pr.pkg.hunter_box_1

Open pull request to ingenue/hunter repository, to pkg.template branch:

Pull request with tests

I will create pkg.hunter_box_1 and change branch before merging:

Change branch

Pull requests

After CI configs merged you can open pull request with package itself:

[hunter]> git checkout pr.hunter_box_1
[hunter]> git push -u origin pr.hunter_box_1
PR with package

At this moment you know the pull request number:

Pull request number

Add it to documentation:

[hunter]> git checkout pr.hunter_box_1
[hunter]> vim docs/packages/pkg/hunter_box_1.rst
[hunter]> git add docs/packages/pkg/hunter_box_1.rst
[hunter]> git commit -m 'Pull request number'
[hunter]> git push

Pull request will be approved and tests run on CI, documentation will be tested automatically:

Package testing

Branch pkg.hunter_box_1.pr-N will be created from pkg.hunter_box_1 to test package:

Change branch

Upload

After tests passed pull request will be merged and upload run. When upload finished new release will be created:

Upload

You can use new URL/SHA1:

Upload

Clean

At this moment all branches can be removed:

[hunter]> git checkout master

[hunter]> git push origin :pr.hunter_box_1
[hunter]> git push origin :pr.pkg.hunter_box_1
[hunter]> git push origin :test.hunter_box_1

[hunter]> git branch -D pr.hunter_box_1
[hunter]> git branch -D pr.pkg.hunter_box_1
[hunter]> git branch -D test.hunter_box_1