App Testing¶

What does it do?¶

The App Testing framework lets you write correctness tests for your Compute apps: pick a set of inputs and assert the app's results against values you know are right — from a code clause, a worked example, or an Excel cell. Tests are plain pytest and run your apps through a local Compute Engine.

How to use?¶

Prerequisites¶

A local Compute Engine running and an API key.
Python 3.11.

Installation¶

Download the Compute SDK and the test framework, and extract both.
Install them into your Python environment (run this from the folder that holds the two extracted folders):
```
pip install ./compute-sdk ./compute-test-framework
```
This installs the compute_sdk and test_framework packages plus pytest — so you can import test_framework and import compute_sdk from anywhere.
Copy conftest.py, pytest.ini, and .env.template from compute-test-framework/ into the root of your collection — the folder you run pytest from.
Copy .env.template to .env and add your API_KEY (and optionally COMPUTE_ENGINE_URL).

Writing a test¶

Put tests under <app>/tests/test_<app>.py. Each test solves the app with the solve fixture, then asserts results with assert_numeric / assert_exact:

from test_framework.assertions import assert_numeric, assert_exact

APP_NAME = "EC2_Concrete_Properties"          # the app's entry class

def test_c30_37(solve):
    results = solve(APP_NAME, {"fck": 30})
    assert_numeric(results["fcm"], 38.0, abs=0.5, source="EN 1992-1-1 Table 3.1")
    assert_exact(results["cracked_state"], "Cracked")

assert_numeric takes exactly one of rel= or abs=; source= optionally records where the expected value came from.
Reading a result the app didn't return raises a clear error listing the available variables.
test_framework/example_test_app.py (in the installed package) is a fuller example.

Danger

Never copy expected values from the engine's own output — that only checks the app against itself. Source them independently: a code clause, a worked example, or an Excel cell.

Running tests¶

pytest collections/<collection>/apps/<app_dir>/tests -v

Info

If the Engine isn't running or .env is missing, tests skip rather than fail — a run that is entirely skipped is not a pass.

Claude Code skills¶

Two Claude Code skills automate authoring and running these tests:

cb-tests-create — writes a test module for an app: it reads the app's inputs, helps you gather independently-sourced expected values, and writes the test (it never runs the tests).
cb-tests-run — runs existing tests against your local Engine and reports the results.

Installing the skills¶

Download the skills here and extract them.
Copy the cb-tests-create/ and cb-tests-run/ folders into your project's .claude/skills/ folder (create it if it doesn't exist).
In Claude Code, run /cb-tests-create to write tests for an app, then /cb-tests-run to run them.