Raw quoted literals

[mattheww]

The FLS models raw string literals as follows, in §2.4.6.2 (fls_usr6iuwpwqqh):

RawStringLiteral ::= r RawStringContent

RawStringContent ::= NestedRawStringContent | " ~[\r]* "

NestedRawStringContent ::= # RawStringContent #

This is close to the form used in the Reference.

Problem 1

This model makes forms like r"a" r"b" ambiguous: there could be a single RawStringLiteral or two RawStringLiterals here (rustc says there are two).

That means a MacroInvocation like m!(r"a" r"b") is underspecified, and a "longest match" principle as suggested in #589 would give the wrong answer.

The ambiguity is still present in forms using hashes, for example r#"a"# r#"b"#.

Worse, this model accepts things that rustc rejects: it (unambiguously) accepts m!(r"a"b").

Similar things are true for raw byte string literals (§2.4.2.2, fls_jps9102q0qfi) and Raw C String Literals (§2.4.3.2, fls_g4ldypf3rl6i).

Problem 2

Rustc (since I think 1.61) has a limit of 255 hash characters on each side of NestedRawStringContent, NestedRawByteStringContent, and NestedRawCStringContent, but the FLS doesn't document that restriction.

See rust-lang/reference#1180.

[mattheww]

The Reference nods in the direction of the first problem by adding a "(non-greedy)" annotation to the * in its equivalent of RawStringContent, but it doesn't define "non-greedy" further.

I don't think an approach based on modifying the meaning of a subexpression of RawStringContent is promising: with a grammar of this shape, if RawStringLiteral accepts r#"a"b"# (as it should) it must be the case that RawStringContent accepts "a"b", but then it seems RawStringLiteral would accept r"a"b".

Maybe there's a reasonable approach based on modifying the meaning of RawStringContent itself, using something like this:

A parsing expression E-annotated-as-minimal accepts input s when:

• E accepts s; and
• E doesn't accept any proper prefix of s.

[starblue]

The start delimiter of the raw string r#..#" determines the end delimiter "#..# (same number of #, possibly 0). So to prevent the ambiguity you could additionally require that the content doesn't contain the end delimiter.