In the previous post, I covered how Pintos Test Explorer runs a Make target and displays the generated test artifacts in a VS Code sidebar.
This time, I want to look at the internal layer shared by the sidebar and terminal workflows.
The project includes two equivalent commands:
pt pintos-tests
For example:
pt list threads pt run threads alarm-zero pt debug threads alarm-zero pt artifacts threads alarm-zero
Both commands use the bundled Python helper in pintos-test-cli.py.
Why a companion CLI was necessary
At first, the project only needed a VS Code sidebar.
However, some operations are easier from the terminal.
Run tests by wildcard Run a numeric range Inspect artifact paths Reset several results Use the test runner in a script
For example:
pt run threads 11-20 pt run threads alarm-* pt reset threads all
Maintaining separate discovery logic for the sidebar and CLI would eventually cause them to show different test lists.
I therefore made the sidebar call the bundled Python helper for test discovery.
VS Code sidebar → pintos-test-cli.py list threads --json → JSON test list → PintosTreeProvider → Sidebar nodes
The terminal command uses the same helper directly.
pt list threads → pintos-test-cli.py → Same discovery logic
The extension and CLI also follow the same project, Make target, and artifact rules when executing tests.
Discovering tests from Makefiles
Pintos tests are not stored in a single JSON file. They are registered through Make.tests files and Make variables.
A simplified registration may look like:
TESTS += tests/threads/alarm-zero TESTS += tests/threads/alarm-single
It may also use Make expressions.
tests/threads_TESTS = alarm-zero alarm-single TESTS += $(addprefix tests/threads/, $(tests/threads_TESTS))
The first implementation asked Make to calculate the test list for each project. This was accurate, but starting Make repeatedly made the sidebar slower.
I then added direct Make.tests parsing.
The parser supports common assignment forms.
= := ?= +=
It also evaluates the Make expressions commonly used by Pintos.
Variable references $(addprefix ...) $(patsubst ...)
Direct parsing made initial discovery much faster, but it introduced another issue.
The test definitions found in Make.tests are not always identical to the final TESTS list selected by a project’s build Makefile.
The current approach combines both sources.
Final TESTS list from the build Makefile → Primary project test list Project-owned Make.tests entries → Supplemental nested or optional tests
This keeps project-level operations aligned with the tests selected by the current build while still exposing optional tests that can be run explicitly.
Keeping tests inside the correct project
Some test names can appear in more than one build context.
For example, User Programs tests may be available while working on Virtual Memory. Without filtering, thread tests such as alarm-* could also appear under the wrong project after evaluating shared Make variables.
Each project therefore owns a specific prefix.
Threads → tests/threads/ User Programs → tests/userprog/ Virtual Memory → tests/vm/ File System → tests/filesys/
The discovery helper filters every result using the project prefix before displaying it.
This keeps the sidebar structure stable even when project Makefiles include shared definitions.
Finding the test source file
The extension provides an Open Test Source action.
The obvious implementation would be:
tests/threads/alarm-zero → tests/threads/alarm-zero.c
But this is not always correct.
A Makefile may explicitly map a test target to another source file through a _SRC variable. Thread tests can also register a test name with a differently named C function inside tests.c.
The helper therefore uses several strategies.
Read the test’s _SRC registration → Check the matching .c path → Inspect thread test registrations → Search candidate source files for the registered symbol → Select the best existing match
This is why a test such as alarm-single can open its real implementation even when the filename and registered function do not match perfectly.
Using the CLI inside VS Code
When the extension activates, it copies the bundled CLI runtime into VS Code’s extension storage.
It then modifies the environment of newly opened integrated terminals.
Add the CLI directory to PATH Set PINTOS_ROOT Set PINTOS_WORKSPACE_ROOT
As a result, a new terminal can immediately run:
pt --help
The Pintos root is pinned to the one discovered by the extension, so moving between directories inside a wrapper workspace does not accidentally read artifacts from another Pintos tree.
For external terminals, the extension can install small wrappers into:
~/.local/bin
Debugging a Pintos test
Running a test and debugging a test are different problems.
A normal test run starts Pintos and waits for the checker result. A debug session must start Pintos in GDB server mode and keep it running while VS Code attaches.
The GDB preparation logic is implemented in pintos-gdb-server.sh.
At first, it might seem enough to run:
gdb kernel.o
But kernel.o does not describe which test should run, which files should be copied into the Pintos file system, or which QEMU arguments are required.
The helper first asks Make to print the real command for the selected test without executing it.
make -n tests/threads/alarm-zero.output
It then extracts the generated pintos command and modifies it for debugging.
Resolve the normal test command → Preserve the selected test and copied files → Add the Pintos --gdb option → Start QEMU’s GDB server → Wait for port 1234 → Notify VS Code that the server is ready
VS Code then starts a cppdbg session with a configuration similar to:
1Program: project/build/kernel.o 2Debugger: gdb 3Server: 127.0.0.1:1234
The resulting flow is:
Click Debug → Prepare the project build tree → Resolve the real Pintos test command → Start Pintos and QEMU in GDB mode → Wait for the GDB server → Attach VS Code C/C++ debugger → Debug the kernel
When the VS Code debug session ends, the extension stops the GDB server and removes its stored process state.
User Programs tests through the VM build
Virtual Memory development introduces another special case.
While implementing VM, I may still want to run User Programs tests. The tests should remain listed under User Programs, but they need to run against the VM kernel.
The extension provides a User Programs for VM checkbox.
Disabled User Programs test → userprog/build Enabled User Programs test → vm/build
The selected source project and execution project are treated separately.
Source project: User Programs Build project: Virtual Memory
Running, debugging, and reading artifacts all use the selected build project, while the test remains in the User Programs section of the sidebar.
Managing custom tests
Creating a custom Pintos test requires more than creating one C file.
Depending on the project, the operation may need to:
1Create the C source 2Create the checker file 3Register the test in Make.tests 4Register a Threads test function 5Create matching build output directories
Renaming or deleting a test must update the same resources.
If only the source file is deleted while its Make.tests registration remains, unrelated builds may fail because Make still expects the missing target.
The custom test commands therefore treat these files as one logical unit.
pt custom create threads custom/alarm/new-test pt custom rename threads custom/alarm custom/alarm-clock pt custom delete threads custom/alarm-clock
The VS Code create, rename, and delete actions call the same bundled helper.
VS Code command → pintos-test-cli.py custom ... → Update source files → Update Make.tests → Update registrations → Move or delete artifacts → Refresh the sidebar
How the project evolved
The current architecture was built in several stages.
0.1.0 - Pintos Activity Bar view - Run and Debug actions - Checkbox-based batch execution - Artifact links 0.1.1 - Bundled discovery and GDB helpers - Support for wrapper repositories 0.1.2 - Faster Make.tests-based discovery - Improved GDB startup errors 0.1.8–0.2.0 - Integrated pt and pintos-tests commands - Installable shell wrappers - Expanded Pintos root discovery 0.2.1–0.2.4 - Custom test management through the shared CLI - Source navigation improvements - Build directory repair - Stable checkbox state during asynchronous discovery 0.2.9 - Discovery aligned with the build’s final TESTS list - Supplemental nested Make.tests support - Consistent artifact and root selection 0.3.0 - User Programs tests through vm/build
The release history is documented in the project’s CHANGELOG.
Conclusion
The final structure became:
VS Code sidebar │ ├─ Bundled Python helper │ ├─ Root discovery │ ├─ Test discovery │ ├─ Source resolution │ ├─ CLI commands │ └─ Custom test management │ ├─ Existing Pintos Make targets │ └─ output / result / errors │ └─ GDB server helper └─ VS Code C/C++ debugger
The extension does not replace Pintos’s build, test, or debugging tools.
Instead, it connects them so that the same test can be discovered, run, inspected, and debugged from either the sidebar or the terminal.
What began as a way to avoid repeatedly typing Make commands became a project about test discovery, build-system parsing, process management, artifact synchronization, and remote kernel debugging.