Some testcases fail and one never finishes

[yurivict]

      Start 10: test-cbor_all
10/78 Test #10: test-cbor_all .......................***Exception: Child aborted  9.98 sec
      Start 56: test-msgpack_all
56/78 Test #56: test-msgpack_all ....................***Exception: Child aborted  9.99 sec
      Start 74: test-ubjson_all
74/78 Test #74: test-ubjson_all .....................***Exception: Child aborted  2.96 sec
      Start 78: test-unicode_all
/// {never finishes}

OS: FreeBSD-11.1 amd64
Version 3.1.2 built from port, while trying to update it.

[nlohmann]

Which compiler are you using? What are the error messages?

[yurivict]

$ c++ --version
FreeBSD clang version 5.0.1 (tags/RELEASE_501/final 320880) (based on LLVM 5.0.1)

Some cases that failed before individually don't print errors for some reason.

$ ./work/.build/test/test-cbor 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test-cbor is a Catch v1.9.7 host application.
Run with -? for options

-------------------------------------------------------------------------------
single CBOR roundtrip
  sample.json
-------------------------------------------------------------------------------
/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-cbor.cpp:1376
...............................................................................

/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-cbor.cpp:1376: FAILED:
due to unexpected exception with message:
  [json.exception.parse_error.101] parse error at 1: syntax error - unexpected
  end of input; expected '[', '{', or a literal

===============================================================================
test cases:       5 |       4 passed | 1 failed
assertions: 1246600 | 1246599 passed | 1 failed

$ ./work/.build/test/test-msgpack 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test-msgpack is a Catch v1.9.7 host application.
Run with -? for options

-------------------------------------------------------------------------------
single MessagePack roundtrip
  sample.json
-------------------------------------------------------------------------------
/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-msgpack.cpp:1135
...............................................................................

/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-msgpack.cpp:1135: FAILED:
due to unexpected exception with message:
  [json.exception.parse_error.101] parse error at 1: syntax error - unexpected
  end of input; expected '[', '{', or a literal

===============================================================================
test cases:       2 |       1 passed | 1 failed
assertions: 1049596 | 1049595 passed | 1 failed


$ ./work/.build/test/test-ubjson
===============================================================================
All tests passed (591507 assertions in 3 test cases)

$ ./work/.build/test/test-unicode 
===============================================================================
All tests passed (351 assertions in 1 test case)

[yurivict]

I got these errors in the FreeBSD port with this patch applied: https://bz-attachments.freebsd.org/attachment.cgi?id=192018

[nlohmann]

The error message reads like an input file could not be found. Can you please make sure you ran the tests as described in https://github.com/nlohmann/json#execute-unit-tests ?

[mjstahlberg]

For me on Arch Linux on an Intel i7-3520M all tests pass but the last one just took an enormous amount of time:

78/78 Test #78: test-unicode_all ....................   Passed  114.18 sec

Maybe you need to wait a littler longer for it to complete?

[nlohmann]

Indeed the Unicode tests take the longest time. It's best to execute them with ctest -VV to see some progress output.

[yurivict]

I do out-of-source build, that is located in /usr/ports/devel/nlohmann-json/work/.build.
To run tests, I do:

cd /usr/ports/devel/nlohmann-json/work/.build &&
cmake -DJSON_BuildTests:BOOL=ON /source/path &&
gmake all &&
gmake test

Is this not the right way to run tests?

[nlohmann]

I'm not a Cmake expert - I am certain this works: https://github.com/nlohmann/json#execute-unit-tests

This is how we do it on Travis and AppVeyor.

[yurivict]

Exact same sequence of commands fail on 2 testcases:

56/78 Test #56: test-msgpack_all ....................***Exception: Child aborted  9.98 sec
56/78 Test #56: test-msgpack_all ....................***Exception: Child aborted  9.98 sec
Assertion failed: (m_parent), function moveToParent, file /usr/ports/devel/nlohmann-json/work/json-3.1.2/test/thirdparty/catch/catch.hpp, line 6222.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test-msgpack is a Catch v1.9.7 host application.
Run with -? for options

-------------------------------------------------------------------------------
MessagePack roundtrips
  input from msgpack-python
  output to output adapters
  std::vector<uint8_t>
  output to output adapters
-------------------------------------------------------------------------------
/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-msgpack.cpp:1377
...............................................................................

/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-msgpack.cpp:1377: FAILED:
  {Unknown expression after the reported line}
with expansion:

due to a fatal error condition:
  filename := "test/data/json_nlohmann_tests/all_unicode.json"
  filename := "test/data/json.org/1.json"
  SIGABRT - Abort (abnormal termination) signal

Assertion failed: (m_parent), function moveToParent, file /usr/ports/devel/nlohmann-json/work/json-3.1.2/test/thirdparty/catch/catch.hpp, line 6222.
      Start 74: test-ubjson_all
74/78 Test #74: test-ubjson_all .....................***Exception: Child aborted  3.00 sec
Assertion failed: (m_parent), function moveToParent, file /usr/ports/devel/nlohmann-json/work/json-3.1.2/test/thirdparty/catch/catch.hpp, line 6222.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test-ubjson is a Catch v1.9.7 host application.
Run with -? for options

-------------------------------------------------------------------------------
UBJSON roundtrips
  input from self-generated UBJSON files
  output to output adapters
  std::vector<uint8_t>
  output to output adapters
-------------------------------------------------------------------------------
/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-ubjson.cpp:2203
...............................................................................

/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-ubjson.cpp:2203: FAILED:
  {Unknown expression after the reported line}
with expansion:

due to a fatal error condition:
  filename := "test/data/json_nlohmann_tests/all_unicode.json"
  filename := "test/data/json.org/1.json"
  SIGABRT - Abort (abnormal termination) signal

Assertion failed: (m_parent), function moveToParent, file /usr/ports/devel/nlohmann-json/work/json-3.1.2/test/thirdparty/catch/catch.hpp, line 6222.

These might be FreeBSD-specific, not sure.

[nlohmann]

In all cases, this is an assertion inside the Catch code. This is not a library issue, but should be reported at https://github.com/catchorg/Catch2 - maybe they know what this failing assertion means.

[nlohmann]

Thanks for reporting!

[nlohmann]

I close this issue as this seems to be an issue in the intersection between Catch and FreeBSD.