Chinese character initialize error

[xingyun86]

Chinese character initialize error

What is the issue you have?

//main.cpp encode=utf8
int main(int argc, char ** argv)
{
	std::cout << "Hello CMake." << std::endl;
	setlocale(LC_ALL, "chs");
	try
	{
		// create object from string literal
		auto jsonStr1 = "{\"name\": \"中国\", \"happy\": true, \"pi\": 3.141 }"_json; // **this is error**
		auto jsonStr2 = R"(
		 {
			"name": "中国",
			"happy": true,
			"pi": 3.141
		 }
		)"_json;// **this is error also**
		{
			json con_json;
			con_json["aaa"] = "张三";//**this is ok**
			std::cout << con_json.dump() << std::endl;
		}
         } catch (const std::exception& e)
        {
              std::cout << e.what() << std::endl;
        }
}

#[error info]:
[json.exception.parse_error.101] parse error at line 3, column 14: syntax error while parsing value - invalid string: ill-formed UTF-8 byte; last read: '"中'

Please describe the steps to reproduce the issue.

1.build ok
2.run error.
3.

Can you provide a small but working code example?

What is the expected behavior?

Normal parse

And what is the actual behavior instead?

con_json["aaa"] = "张三";//this is ok
auto jsonStr1 = "{"name": "中国", "happy": true, "pi": 3.141 }"_json; // this is error
auto jsonStr2 = R"(
{
"name": "中国",
"happy": true,
"pi": 3.141
}
)"_json;// this is error also

Which compiler and operating system are you using?

• Compiler: msvc_vs2019
• Operating system: win10v2004_

Which version of the library did you use?

• [ x] latest release version 3.9.0
• other release - please state the version: ___
• the develop branch

If you experience a compilation error: can you [compile and run the unit tests]

• yes
• [ x] no - please copy/paste the error message below

[nlohmann]

Please make sure the file is UTF-8 encoded.

[xingyun86]

I solve it ok now. Thanks.

[leielyq]

I solve it ok now. Thanks.

请问老哥 咋解决的？俺这个代码也是utf-8的，但是中文乱码了呀

[smily1223]

I solve it ok now. Thanks.

老哥怎么解决中文的问题的？

[falbrechtskirchinger]

English, please. We can't help you if we can't understand you.

Answering your translated question…

Either make sure your source file is UTF-8 encoded or use u8 string literals, i.e.

json j = json::parse(u8"JSON string with Chinese characters");
// or
json j = json::parse(u8 R"(JSON string with Chinese characters)");

https://en.cppreference.com/w/cpp/language/string_literal

[wkingnet]

finally I solve this problem

Other UTF8 characters such as Russian, Greek and special symbols also can be displayed correctly.
tutorial by chinese: https://wkings.blog/archives/1184

Features:

1. chinese string in json object.
2. save json file as UTF-8 with not BOM.
3. read utf-8 json file as json object.
4. console display chinese as usual.

tutorial:

step 1

visual studio set source and execution character sets to UTF-8

step 2

visual studio menu - file - Advanced Save Options - set "UNICODE（UTF-8 with BOM） - 65001"

step 3

add code at top of Main(). This will display UTF8 correctly.

  // open and set console start
  AllocConsole();

  system("chcp 65001");  //set console character set
  CONSOLE_FONT_INFOEX info = { 0 };
  info.cbSize = sizeof(info);
  info.dwFontSize.Y = 16;  // or you want font size
  info.FontWeight = FW_NORMAL;
  wcscpy_s(info.FaceName, L"Consolas");  // or you want font
  SetCurrentConsoleFontEx(GetStdHandle(STD_OUTPUT_HANDLE), NULL, &info);

  freopen_s((FILE**)stdout, "CONOUT$", "w", stdout);
  freopen_s((FILE**)stdout, "CONOUT$", "w", stderr);
  freopen_s((FILE**)stdout, "CONIN$", "r", stdin);
  std::cout.clear();
  std::clog.clear();
  std::cerr.clear();
  std::cin.clear();

  // std::wcout, std::wclog, std::wcerr, std::wcin
  HANDLE hConOut = CreateFile(_T("CONOUT$"), GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
  HANDLE hConIn = CreateFile(_T("CONIN$"), GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
  SetStdHandle(STD_OUTPUT_HANDLE, hConOut);
  SetStdHandle(STD_ERROR_HANDLE, hConOut);
  SetStdHandle(STD_INPUT_HANDLE, hConIn);
  std::wcout.clear();
  std::wclog.clear();
  std::wcerr.clear();
  std::wcin.clear();
  //  open and set console end

this is my utf-8 json test code.

#include <iostream>
#include <fstream>
#include <nlohmann/json.hpp>
using namespace std;
using json = nlohmann::json;
  json j = {
    {"read",{
      {"player_id", {
        {"addr", 0x011B90C4},
        {"offest", {}},
        {"comment", "游戏角色的ID"},
      }},
      {"player_name", {
        {"addr", 0x011B90B8},
        {"offest", {0x10,0x20,0x30}},
        {"comment", "γηυ测试國ふそ▁☒☑■☾•♫¶¤φÇÏÅはにてヌホム㉢ㅆㅔㅃㅋㅝㅘㅎЩОёыфцэыъшмбгжф"},
      }},
    }},
  };
  try {
    ofstream o("pretty.json");
    o << std::setw(0) << j << std::endl;
    // read a JSON file
    std::ifstream k("pretty.json");
    json l = json::parse(k);
    json read = l["read"];
    cout << read["player_name"] << endl;
  }
  catch (exception& e) {
    cout << e.what() << endl;
  }

[nlohmann]

FYI: An UTF-8 BOM is ignored by the library - you don't need to remove it.

[wkingnet]

FYI: An UTF-8 BOM is ignored by the library - you don't need to remove it.

honestly I don't know how to set the BOM. When Visual Studio Compiler command with not /utf-8, file pretty.json will be save as "GB2312". when with /utf-8, file pretty.json will be save as "UTF-8 with not BOM".

[nlohmann]

If it works, then everything is fine.