Handled UTF-8 BOM

[FrankHB]

Fixed #1155 .

[ehuss]

Thanks for the PR! Can you also add a test for this?

[ehuss]

Just to avoid the extra allocation, perhaps this could be content.replace_range(..3, "");

[FrankHB]

Not sure where should be the test. Is an additional BOM in tests/dummy_book/src/first/unicode.md enough?

[ehuss]

I'm a little concerned about having invisible bytes in a file. unicode.md is already quite subtle. But I think if you have an editor that can't handle a BOM properly, it will have a risk of corrupting the existing unicode, so it's probably fine.

My only other idea is to create a simple 1-chapter book in a temp directory like book_with_a_reserved_filename_does_not_build does. Whichever seems easiest!

[FrankHB]

Sorry, I don't get the point of the temp directory.

As of the subtleness... I largely agree. Nevertheless, I can use a hex editor to make sure only the BOM is inserted exactly. Anyway, the visible part of the file has already warned us about this. (Or one more visible warning about invisible bytes?)

[ehuss]

Creating a book in Rust code (and writing it to a temp directory) would allow writing the content in a Rust string where the characters can be escaped and explicit. For example, fs::write(chapter, "\u{feff}# Example chapter").

[ehuss]

Thanks!