Reporting test coverage for Swift packages without Xcode

by Michele Volpato

August 11th, 2020

Learn how to report test coverage of Swift packages without using Xcode. By running tests in a Linux machine, you can integrate the process in your CI/CD system using a Docker container.

If you use Xcode, Swift packages are the easiest way to include reusable components of Swift code in your project. You can easily add a Swift package to your project using Xcode, which uses the Swift Package Manager to manage dependencies automatically. 

Adding Alamofire to your Xcode project via the Swift Package Manager.

Writing your own Swift packages is a good way of organizing your code base in reusable and easy to maintain libraries. 

Think about an API that you access from more than one of your apps. Rewriting a Swift wrapper for each of your projects, without sharing code among them, is not only a waste of work, but also a way to add confusion to your team, which is now switching between different implementations of the same concept. It increases the chance of bugs in your code. 

By moving the common code into a Swift package, it becomes the only source of truth for that code, making it easier to maintain.

Testing a Swift package

You can work on a package directly in Xcode, and you can write tests and test your package, as you do with your apps. It is also possible to collect coverage data, using Xcode.

Testing a Swift package in Xcode. Collecting code coverage while testing a Swift package in Xcode.

But you do not need Xcode, or even a mac, to work on and test a Swift package. In fact, the Swift ecosystem is open source, and it can be used on Linux to build and test Swift libraries and application. 

In case you are not using Xcode, you need a different way to collect coverage information about your test suite, and that is exactly what I will show you in the rest of this article.

Enable test coverage

Test coverage information can be gathered easily. Open your terminal, or console, and change directory to the one containing your package. Then run swift test --enable-code-coverage. After tests are executed, you are greeted with an output like

Testing a Swift package from Terminal.

But where is the report about coverage? The swift test command does not specify it:

The output of swift test --help, on terminal.

You need to search for it, I found mine in [PROJECT_FOLDER]/.build/x86_64-apple-macosx/debug/codecov, and it is in a json format.

Change the output format

If you prefer a different output format, unfortunately the current version of the swift test command does not allow you to change it. But the tool it uses might allow it.

In fact, it does. If we run

swift test --enable-code-coverage --verbose

we notice that it uses the export command of LLVM-cov.
According to its documentation, llvm-cov export has a -format option:

-format=FORMAT Use the specified output format.

The supported formats are: text (JSON), and lcov. We can run the same command swift test uses, and change the format:

[PATH_TO_LLVM-COV]/llvm-cov export \ 
    -format=lcov \ 
    -instr-profile=.build/x86_64-apple-macosx/debug/codecov/default.profdata \ 
    .build/x86_64-apple-macosx/debug/[TEST_PACKAGE_NAME].xctest/Contents/MacOS/[TEST_PACKAGE_NAME] \ 
    > lcov.info

You can find the correct paths for you when you run swift test --enable-code-coverage --verbose. You also might need to install a version of LLVM-cov.

Generate reports for your boss

From the lcov.info file you exported in the previous section, you can generate a collection of HTML pages to show to your boss. You will need to install lcov first. On a mac you can use Homebrew: brew install lcov and then you can run

genhtml lcov.info --output-directory ./coverage/

which produces a 2003 looking website: 

Code coverage report generated with lcov version 1.14

What to do with the report

We use the result from the code coverage report to update our GitLab project, both for the merge request

A merge request in GitLab with a code coverage value

and for the project itself 

The code coverage badge at the project page in GitLab

and, because we do not need Xcode or a macOS machine to collect the information, we can do everything from a Docker container launched from GitLab CI/CD, or using an external CI/CD service, like Bitrise.

Read more

If you want to know more about test coverage for Swift packages, the Swift Dev provides some additional options to the content of this blogpost, while BrightDigit extends the topic to integrating CodeCov.io.

Do you want to know more about our expertise in Swift? Read why we are using Vapor as a server side Swift framework over Kitura.