Skip to content

[3.0][Testing] Case insensitive comparisons (part 2 of 2) — guard the convention in the unit suite - #9597

Merged
jdarwood007 merged 5 commits into
SimpleMachines:release-3.0from
albertlast:3.0/tests-ci-token-guard
Oct 4, 2026
Merged

jdarwood007 merged 5 commits into
SimpleMachines:release-3.0from
albertlast:3.0/tests-ci-token-guard

Conversation

@albertlast

@albertlast albertlast commented Aug 31, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Part 1 (#9596) gives a comparison a way to say whether it folds case. This is what notices when
one does not say. What makes it worth a test rather than a review habit is the failure
mode: a case sensitive comparison returns fewer rows instead of erroring, so nothing
reaches smf_log_errors, nothing fails in CI, and the first person to notice is a member
whose search came back empty. #9592 and #9593 both sat in the tree for years.

There are two ways to get this wrong, so there are two scans.

1. Folding neither side. The comparison relies on the collation, which folds on MySQL
and does not on PostgreSQL. Counted per file against a baseline:

public const BASELINE = [
    'Sources/Actions/Admin/Bans.php' => 1,
    'Sources/Actions/Admin/Subscriptions.php' => 1,
    'Sources/Actions/Profile/Summary.php' => 2,
    'Sources/Search/SearchApi.php' => 3,
    'Sources/Security.php' => 2,
];

Three are #9593. The two in Security.php are folded by the ban_like identifier when the
query runs rather than in the query text, and the one in Bans.php compares
email_address_ci against a ban's email pattern, which is stored casefolded. Those three
stay listed with a note saying why.

This scan reads LIKE only. member_name = {string:name} and $member_name = $string
are the same line to a scanner, and an UPDATE ... SET clause looks like both; guessing
would produce a baseline of about fifty entries that are mostly PHP assignments, which is
a test people learn to ignore.

2. Folding only the column. This is the worse one, and it is why the second scan
exists. A folded column can never equal an unfolded value, so instead of matching too much
the comparison matches nothing at all on PostgreSQL — including the row it was looking
for. It also hides, because the query says {column_ci:} and reads as handled. The first scan
cannot find these by construction: it treats {column_ci:} as proof a decision was made.

public const UNFOLDED_VALUES = [
    'Sources/Actions/Admin/Members.php' => 1,
    'Sources/Actions/AutoSuggest.php' => 2,
    'Sources/Actions/RequestMembers.php' => 1,
    'Sources/PersonalMessage/PM.php' => 1,
    'Sources/PersonalMessage/Search.php' => 2,
];

These are correct only when the caller folds the value in PHP first, which is a claim
about code elsewhere, so the list names which callers do and which do not. Five do.
Two do not, and are a fault: PersonalMessage/Search.php, in the search for a personal
message by its author, which is #9598.

This scan reads equality as well as LIKE, which the first cannot. A line carrying
{column_ci:} is SQL, so an = on it is a comparison rather than an assignment — which is how
#9594 came into range before #9633 fixed it.

Sources/User.php left this list when #9596 gained {array_string_ci:}, which folds every
value in an IN list, so that comparison folds both sides now.

Confirmed to fail when it should. Adding 'WHERE real_name LIKE {string:x}' to
Sources/Actions/Groups.php:

The set of case sensitive comparisons on user text has changed.
...
     'Sources/Actions/Admin/Subscriptions.php' => 1,
+    'Sources/Actions/Groups.php' => 1,
     'Sources/Actions/Profile/Summary.php' => 2,

It also fails when a count goes down, which is what happened on the rebase: #9633 moved the
email comparisons in Bans.php, Register2.php and Profile.php onto email_address_ci,
and the test asked for its baselines to be lowered to match. That is the last commit here.

A third test asserts both scans find something, so that emptying either one does not leave
the tests above passing against an empty baseline.

Neither scan asserts anything about behaviour, so this is not a substitute for the fixes.
It stops the lists growing while they are worked through, and lowering a number is the
point of them rather than a chore attached to them.

Reading every file under Sources/ costs about 60ms on a normal filesystem.
It is slower over a Docker bind mount on Windows, which is a local artifact.

Issues References (Fixes|Related|Closes)

  1. Related: [3.0] Case insensitive comparisons (part 1 of 2) — a {ci:} type for the query language #9596 — the {column_ci:} type this guards the use of
  2. Related: [3.0]: Searching by user name is case-sensitive on PostgreSQL #9593
  3. Related: [3.0]: Searching personal messages by author finds nothing on PostgreSQL #9598
  4. Related: [3.0]: Email bans do not match members with uppercase in their address on PostgreSQL #9592, [3.0]: Two accounts can register on the same email address on PostgreSQL #9594 — fixed by [3.0] Improves handling of case sensitivity for email addresses #9633, which lowered the baselines here
  5. Related: Enable an automated AI reviewer on pull requests (CodeRabbit is free for public repos) #9526 — the same gap this covers mechanically, where an AI reviewer would cover it by judgement

🤖 Generated with Claude Code

@jdarwood007

Copy link
Copy Markdown
Member

Can this get a rebase?

@jdarwood007 jdarwood007 added this to the 3.0 Alpha 5 milestone Oct 4, 2026
albertlast and others added 5 commits October 4, 2026 16:54
A LIKE against text a person typed has to say whether it folds case, because
the engines disagree about it. Written as a bare column it says nothing, and
the query then matches on MySQL and not on PostgreSQL. That failure returns
fewer rows instead of erroring, so it reaches neither the error log nor CI.

This counts, per file, the comparisons on the columns holding names, email
addresses and hostnames that do not fold case in the query text, and holds the
count against a baseline. A new one fails the suite and names the file; fixing
an old one means lowering its number in the same commit.

The scan covers LIKE only. `member_name = {string:name}` and
`$member_name = $string` are the same line to a scanner, and the SET clause of
an UPDATE looks like both, so equality is left out rather than guessed at.

The two comparisons in Security.php are folded by the 'ban_like' identifier at
the point the query runs rather than in the query text, so they stay in the
list with a note saying why.

Reading every file under Sources costs about 60ms on a normal filesystem.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
Folding one side is worse than folding neither. A folded column can never equal
an unfolded value, so rather than matching too much on PostgreSQL the comparison
matches nothing at all, including the row it was looking for. It is also the
harder one to see, because the query says {ci:} and reads as handled.

The first scan cannot find these, by construction: it treats {ci:} as proof that
a decision was made. So a second scan lists the comparisons that fold a column
while comparing it against a value the query leaves alone. Those are correct
only when the caller folded the value in PHP, which is a claim about code
elsewhere, so the list names which callers do and which do not.

This one reads equality as well as LIKE. A line carrying {ci:} is SQL, so an =
on it is a comparison rather than a PHP assignment, which is what put equality
out of reach of the first scan.

Five of the eleven entries are known faults: Register2.php and Profile.php are
SimpleMachines#9594, and PersonalMessage/Search.php is the same fault in the search for a
personal message by its author.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
{ci_string} became {string_ci} and gained {array_string_ci}, so the scan that
decides whether a comparison declares its case handling has to recognise both.

User.php leaves the second list as a result. It hands its names to
{array_string_ci:}, which folds every value in the list, so the comparison no
longer folds one side only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
{ci:} became {column_ci:}, so the scan that decides whether a comparison
declares its case handling has to look for the new spelling. Getting this wrong
fails loudly rather than quietly: every converted call site would stop looking
folded at once, and the baseline would grow by all of them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
SimpleMachines#9633 moved the email comparisons in Bans.php, Register2.php and Profile.php
onto the casefolded email_address_ci column, which closed SimpleMachines#9592 and SimpleMachines#9594.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
@albertlast
albertlast force-pushed the 3.0/tests-ci-token-guard branch from bd84584 to 2143b6c Compare October 4, 2026 14:59
@albertlast

Copy link
Copy Markdown
Collaborator Author

Rebased onto release-3.0. Part 1 (#9596) has merged, so it has dropped out of the diff, and the four commits that remain replayed without conflicts.

There is one new commit on top. The guard test failed after the rebase, and in the direction it is meant to: #9633 moved the email comparisons in Bans.php, Register2.php and Profile.php onto email_address_ci, so both counts went down. The last commit lowers the baselines to match. The one comparison left in Bans.php matches email_address_ci against a ban's email pattern, which is stored casefolded, so it stays listed with a note.

The unit suite passes and php-cs-fixer is clean on the file. I've updated the description to match.

@jdarwood007
jdarwood007 merged commit a1325d2 into SimpleMachines:release-3.0 Oct 4, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants