Test framework choice: revive tut, or doctest / Catch2 / GoogleTest #1244
Labels
No labels
amd
avatars
bug
builder
cannot-reproduce
crash
epic
feature
from-viewer
gl-removal
known-limitation
monitoring
parity
perf
platform
priority
critical
priority
high
priority
low
priority
medium
provenance
rendering
rt
status
in-progress
status
testing
status
up-next
triage
ui
upstream
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
sky_line/Slipstream#1244
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Part of #1243.
Decision to make. Which framework runs the new unit tests; what happens to the existing tut tests.
Recommendation: doctest (single vendored header), with the tut tests left under
LL_TESTSuntil individually ported or deleted.test.cpp/lltut.cpp/APR in every executableDOCTEST_CONFIG_IMPLEMENT_WITH_MAINin one TUgtest+gmocklibs, heavier headersdoctest_discover_tests, or one ctest entry per executable plus--list-test-cases; JUnit / xml reporterscatch_discover_testsgtest_discover_tests-fno-exceptionsif ever needed)Reasons in short: the goal is a gate that adds seconds, not minutes, to every build, so compile and link cost per test matters more than feature count; a single vendored header removes a network dependency from configure (the 3p cache already is a pain point); doctest's
TEST_CASE/SUBCASEstyle is enough for table-driven maths and parser tests; and it can be dropped into any module without the APR-based runner. GoogleTest is the right answer only if we later want death tests or gmock; Catch2 v3 has no advantage over doctest here and costs compile time. Reviving tut means keeping a 2008 library and a runner that is wired to APR and LL singletons, with no IDE story.Existing tut tests. Do not mass-port. Keep
LL_TESTS(tut) buildable as an opt-in-DLL_TESTS=ONbuild, fix it once so it compiles (spike below), and port a suite to doctest only when it covers something worth running every build (the llcommonllsdserialize,llstring,lluri,llkeybindtests are the likely ones; the 25indra/test/*_tut.cppllmessage / http-node suites mostly are not). Anything that stays red and unowned after the spike is deleted, not left.Scope.
-DLL_TESTS=ONon Linux, record which build and which pass; write the result in the ticket. Fix or delete accordingly.doctest.hunderindra/slcore/test/third_party/doctest/with its LICENSE; addindra/cmake/SlTest.cmakewithsl_add_unit_test(<name> SOURCES ... LIBS ... LABELS fast)(one executable per module,add_test, labels, timeout 10 s per test by default).llstringandlluriorllsdserialize), delete their tut copies.fastlabel if the sum exceeds the budget.Acceptance criteria. The decision and the table are recorded in
doc/DEVELOPER_GUIDE.md;sl_add_unit_testworks under GCC (Linux) and clang-cl (thewindows-crosspreset compiles it); two ported suites pass in under 1 s each; the spike's findings on the tut build are written down (builds / does not build, what was fixed or removed).Effort. 3 to 4 days including the spike.