Skip to content

boards/mps2-an500: Add citest configurations; tools/ci: Do not build lm3s6965-ek. - #21972

Open
casaroli wants to merge 3 commits into
apache:masterfrom
casaroli:mps2-an500-citest-up
Open

casaroli wants to merge 3 commits into
apache:masterfrom
casaroli:mps2-an500-citest-up

Conversation

@casaroli

@casaroli casaroli commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

depends-on: /pull/21975

Summary

CI builds the protected configurations (CONFIG_BUILD_PROTECTED=y), but it runs none of them in QEMU. This PR adds two configurations for the Cortex-M7 (ARMv7-M) that CI runs with NTFC, as it runs sabre-6quad:citest and qemu-armv8a:citest: mps2-an500:citest (flat build) and mps2-an500:citest_protected (protected build).

citest is mps2-an500:nsh and citest_protected is mps2-an500:knsh, with the programs that the NTFC cases use: ostest, hello, helloxx, pipe, getprime, heap, setlogmask and the pipe driver. Both have full optimization and no scheduler info messages. citest has no profiling (PROFILE_ALL, PROFILE_MINI, SYSTEM_GPROF).

For citest_protected, QEMU loads the kernel (nuttx) with -kernel and the user image (nuttx_user.elf) with the generic loader (-device loader). NTFC finds the commands in the ELF file of elf_path. In a protected build the NSH commands and the test programs are in the user image, thus elf_path is nuttx_user.elf. With elf_path: './nuttx', NTFC selects 3 of the 933 cases; with nuttx_user.elf it selects 77.

session.json excludes Nuttx_System_Arch_Os_Stability in both configurations. That module runs the heap test 10 times (about 170 s on this board). Nuttx_System_Mm_Integration runs the heap test one time.

run.sh is the one of sabre-6quad:citest, with one change: it moves each NTFC result directory into the artifact directory, not the result directory itself. When testbuild.sh retries a configuration, the result directory of the first attempt is already in the artifact directory. Then mv result fails, result/ stays in the tree, and the tree check fails the retry. All other citest scripts have this problem.

lm3s6965-ek breaks CI often. Since August 2024, 15 commits in boards/arm/tiva/lm3s6965-ek only make a configuration fit or link again, 11 of them since July 2026: 7e0db94, 76b58e4, 2afdcfb, 0a0af89, a83c844, 3f38ca2, 1987473, 31a74e3, 58f89fa, 796fb3c, 6feaf0e, cb6d987, d9d9d3f, f3b6cf8, 51da55d, and now #21962.

Apart from these and tree-wide changes, the board got two commits since 2024: 1128f6b (make qemu-protected boot in QEMU, March 2026) and 9d60832 (enable NET_SOCKOPTS in qemu-protected). qemu-protected again stops at boot today.

CI only builds it and never runs it: lm3s6965-ek:qemu-protected stops after the boot letters ABCDEF in QEMU and gives no NSH prompt, and its 128 KiB kernel flash fails PRs that do not change its code (for example #21942, x86_64 code only, fails with region `kflash' overflowed by 8 bytes).

TI lists the LM3S6965 as Not Recommended for New Designs and recommends TM4C devices for new designs: https://www.ti.com/product/LM3S6965

The new mps2-an500:citest (flat) and mps2-an500:citest_protected configurations cover the same ground and more: ARMv7-M (Cortex-M7), a flat and a protected build, 4 MB of flash and 8 MB of RAM for each side of the protected build, and CI runs NTFC on both in QEMU.

The third commit removes lm3s6965-ek from the CI build list: tools/ci/testlist/arm-ti.dat gets the line -,lm3s6965-ek:.+, the same form as -,arty_a7:.+ in risc-v-04.dat. The two new configurations replace it. No other testlist, workflow or CI script builds or runs lm3s6965-ek (grep of tools/ci and .github: only arm-ti.dat through /arm/tiva, and CODEOWNERS).

The second commit adds both configurations to the board documentation.

Impact

CI only. The Linux (arm-other) job builds the two new configurations with no testlist change (/arm/mps/mps2* in tools/ci/testlist/arm-other.dat), and testbuild.sh -R runs their run.sh.

CI time added to the arm-other job (measured, see Testing): about 5.3 minutes for citest and 4.6 minutes for citest_protected.

CI time removed from the arm-ti job: the 8 lm3s6965-ek configurations took 415 s of the 3112 s of configuration builds in the arm-ti job of a recent CI run (run 44042631731).

The other tiva boards stay in the build (see Testing). No change to the kernel, to the board code or to other configurations.

Make build only. The CMake build of this board has no rule for the user image (No SOURCES given to target: nuttx_user), the same as mps2-an500:knsh; CI builds the mps2 configurations with make.

Testing

Runs as CI does them, on an x86_64 Linux host in the CI image ghcr.io/apache/nuttx/apache-nuttx-ci-linux (podman, 4 CPUs): pip install ntfc==0.0.3, nuttx-ntfc-testing tag release-0.0.2, then ./cibuild.sh -c -A -N -R -S on a testlist with one line (/arm/mps/mps2-an500/configs/<config>,CONFIG_ARM_TOOLCHAIN_GNU_EABI), on nuttx master with this branch and nuttx-apps master. Each run starts with an empty ccache.

Configuration Run NTFC result NTFC time Time for the configuration (clean, configure, build, NTFC, normalize)
citest 1 60 passed, 7 skipped, 0 failed 263 s 316 s
citest 2 60 passed, 7 skipped, 0 failed 260 s 311 s
citest_protected 1 60 passed, 7 skipped, 0 failed 212 s 273 s
citest_protected 2 60 passed, 7 skipped, 0 failed 213 s 277 s

ostest gives ostest_main: Exiting with status 0 in both builds; the flat build also runs the FPU test. The 7 skipped cases are the same as in sabre-6quad:citest (ls -s, uname -v/-m/-i, mw, exec, mmleak). The normalize step (refresh.sh) passes.

In one earlier run of citest_protected (host load about 25), NTFC stopped with device boot timeout before the first test, and the next two retries failed because of the run.sh problem above, with 60 passed and 0 failed in NTFC. The cause of the boot timeout is not known: 80 direct boots of the two images reached nsh> in less than 0.2 s, and 30 more NTFC runs of citest_protected passed.

tools/testbuild.sh -p (list only) on arm-ti.dat, before and after the change: 74 configurations listed in both. Before: 3 skipped (eagle100:thttpd, launchxl-cc1310:nsh, tms570ls31x-usb-kit:nsh). After: 11 skipped, the same 3 and the 8 lm3s6965-ek configurations (discover, nsh, nx, qemu-flat, qemu-kostest, qemu-nxflat, qemu-protected, tcpecho). No other line changes, so the other tiva boards (lm3s6432-s2e, lm3s8962-ek, ekk-lm3s9b96, ...) are still built.

make savedefconfig gives the committed defconfig files. tools/checkpatch.sh -c -u -m -g passes on the three commits.

CI runs both in QEMU with NTFC: mps2-an500:citest is a flat build and
mps2-an500:citest_protected is a protected build
(CONFIG_BUILD_PROTECTED=y), both for the Cortex-M7.  CI builds the
protected configurations, but it runs none of them.

citest is mps2-an500:nsh and citest_protected is mps2-an500:knsh, with
the programs that the NTFC cases use: ostest, hello, helloxx, pipe,
getprime, heap and setlogmask, and the pipe driver.  Both have full
optimization and no scheduler info messages.  citest has no profiling
(PROFILE_ALL, PROFILE_MINI, SYSTEM_GPROF).

For citest_protected, QEMU loads the kernel (nuttx) with -kernel and
the user image (nuttx_user.elf) with the generic loader.  NTFC finds
the commands in the ELF file of elf_path.  The NSH commands and the
test programs are in the user image, thus elf_path is nuttx_user.elf.

run.sh is the one of sabre-6quad:citest, with one change: it moves
each NTFC result directory into the artifact directory, not the result
directory itself.  When testbuild.sh retries a configuration, the
result directory of the first attempt is already there: "mv result"
then fails, result/ stays in the tree, and the tree check fails the
retry.

session.json excludes Nuttx_System_Arch_Os_Stability, which runs the
heap test 10 times (about 170 s).  Nuttx_System_Mm_Integration runs
the heap test one time.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Add a Configurations section with mps2-an500:citest and
mps2-an500:citest_protected and the QEMU commands that load their
images.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Exclude all lm3s6965-ek configurations from the arm-ti test list.
mps2-an500:citest and mps2-an500:citest_protected replace them in CI.
The pull request gives the reasons.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
@casaroli
casaroli requested a review from raiden00pl as a code owner November 17, 2026 18:43

-eagle100:thttpd
-launchxl-cc1310:nsh
-,lm3s6965-ek:.+

@xiaoxiang781216 xiaoxiang781216 Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we remove the broken config directly instead?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should consider a cleanup follow-up PR

@github-actions github-actions Bot added Area: CI Size: M The size of the change in this PR is medium Board: arm labels Nov 17, 2026
@github-actions

Copy link
Copy Markdown

MemBrowse Memory Report · Explore what changed →

No memory changes detected for:


Catch memory regressions in CI with MemBrowse. Free for open source.

@casaroli
casaroli marked this pull request as draft November 18, 2026 00:37
@casaroli
casaroli marked this pull request as ready for review November 18, 2026 00:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: CI Board: arm Size: M The size of the change in this PR is medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants