Skip to content

fix(DataMapper): Preserve Unknown mapping item#3042

Merged
igarashitm merged 1 commit intoKaotoIO:mainfrom
igarashitm:2716-unknown
Mar 18, 2026
Merged

fix(DataMapper): Preserve Unknown mapping item#3042
igarashitm merged 1 commit intoKaotoIO:mainfrom
igarashitm:2716-unknown

Conversation

@igarashitm
Copy link
Copy Markdown
Member

@igarashitm igarashitm commented Mar 17, 2026

Fixes: #2716

When deserializing XSLT, preserve XSLT control elements that is not recognized by DataMapper as an unknown item, keep the DOM fragment as it is so that it could be written back when DataMapper generate the XSLT on serialization

It is shown as unknown item with yellow label
Screenshot From 2026-03-17 17-18-36

When the unknown label is clicked, the unknown part of the XML snippet is shown in tooltip
Screenshot From 2026-03-18 09-00-54

Summary by CodeRabbit

Release Notes

  • New Features
    • Unsupported XML elements are now recognized and displayed with visual warning indicators in data mappings
    • Users can inspect unrecognized elements via a popover to view their XML content
    • Unknown elements are properly preserved when saving and loading mappings

@igarashitm igarashitm requested a review from a team March 17, 2026 21:22
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 17, 2026

📝 Walkthrough

Walkthrough

This pull request introduces support for handling unsupported XSLT elements in the DataMapper. It creates new UnknownMappingItem and UnknownMappingNodeData types to capture, preserve, and display unrecognized XML elements. These elements are visualized in the UI with a warning label and can be round-tripped through serialization without data loss.

Changes

Cohort / File(s) Summary
Unknown Mapping Models
packages/ui/src/models/datamapper/mapping.ts, packages/ui/src/models/datamapper/visualization.ts
Introduces UnknownMappingItem class extending MappingItem with element cloning and UnknownMappingNodeData class extending MappingNodeData for visualization representation.
Serialization & Deserialization
packages/ui/src/services/mapping-serializer.service.ts, packages/ui/src/services/mapping-serializer.service.test.ts
Adds handling for UnknownMappingItem in serialize/deserialize paths, including element cloning during population and default case creation for unknown elements. Tests verify cloning, round-tripping, and ordering behavior.
Visualization Service
packages/ui/src/services/visualization.service.ts, packages/ui/src/services/visualization.service.test.ts
Introduces formatXml() method, updates node creation to use factory pattern, adds UnknownMappingItem filtering logic, and extends menu/selector exclusions. Tests verify XML formatting and unknown node visibility.
UI Components
packages/ui/src/components/Document/NodeTitle/NodeTitle.tsx, packages/ui/src/components/Document/NodeTitle/UnknownMappingLabel.tsx, packages/ui/src/components/Document/NodeTitle/NodeTitle.test.tsx, packages/ui/src/components/Document/AddMappingNode.tsx, packages/ui/src/components/Document/SourceDocumentNode.tsx, packages/ui/src/components/Document/TargetDocumentNode.tsx
Adds new UnknownMappingLabel component rendering warning label with XML popover. Updates NodeTitle to dispatch to unknown label renderer. Adjusts import paths and draggability logic to handle unknown items.
Test Stubs & Examples
packages/ui/src/stubs/datamapper/data-mapper.ts, packages/ui/src/stubs/datamapper/xml/UnknownApplyTemplate.xsl
Adds getUnknownApplyTemplateXslt() function and sample XSLT stylesheet demonstrating an unsupported apply-templates element for testing.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

  • lordrip

Poem

🐰 A field of XSLT unknown,
Now marked with yellow, brightly shown,
We gather what we can't translate,
And round-trip back in pristine state,
Unknown no more—preserved and safe! 🌟

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and concisely summarizes the main change: introducing preservation of unknown mapping items in DataMapper during XSLT processing.
Linked Issues check ✅ Passed The PR fully implements the linked issue #2716 requirements: UnknownMappingItem and UnknownMappingNodeData classes created, UI displays unknown items with yellow labels, raw XSLT elements preserved for serialization, and deletion via trashbox icon is supported.
Out of Scope Changes check ✅ Passed All changes are directly related to the linked issue #2716 objectives. Import path updates and supporting components are necessary infrastructure for the unknown mapping feature implementation.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/ui/src/services/mapping-serializer.service.ts (1)

146-152: ⚠️ Potential issue | 🟠 Major

Unknown items may be reordered during serialization, changing XSLT behavior.

After Line 147 introduces unknown mappings into serialization, they still pass through global sort (Line 74 / Line 151). That can move unknown instructions relative to siblings, violating “preserve and write back as-is.”

🛠️ Suggested safeguard in sort comparator
 private static sortMappingItem(left: MappingItem, right: MappingItem) {
+  if (left instanceof UnknownMappingItem || right instanceof UnknownMappingItem) {
+    // keep insertion order for unknown fragments
+    return 0;
+  }
   const leftFields =
     left instanceof FieldItem ? [left.field] : MappingService.getConditionalFields(left as ConditionItem);
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/ui/src/services/mapping-serializer.service.ts` around lines 146 -
152, The global sort (MappingSerializerService.sortMappingItem) is reordering
UnknownMappingItem instances introduced by
MappingSerializerService.populateUnknownMappingItem, violating the "preserve and
write back as-is" requirement; update the sort comparator (sortMappingItem) to
detect when either operand is an UnknownMappingItem (or both) and return 0 to
preserve original sibling order for unknown items (i.e., do not change relative
order of UnknownMappingItem vs siblings), then continue calling
MappingSerializerService.populateMapping as before for mapping.children so
unknown items are written back unchanged.
🧹 Nitpick comments (1)
packages/ui/src/services/mapping-serializer.service.test.ts (1)

405-420: Round-trip test should assert unknown subtree preservation, not only @select.

Line 419 currently checks one attribute, but the objective is preserving the full unknown fragment.

✅ Suggested test enhancement
   it('should round-trip UnknownMappingItem verbatim', () => {
@@
       const applyTemplates = xsltDocument
@@
         .iterateNext();
       expect(applyTemplates?.nodeValue).toEqual('/ns0:ShipOrder/Item');
+
+      const sort = xsltDocument
+        .evaluate(
+          '/xsl:stylesheet/xsl:template/ShipOrder/xsl:apply-templates/xsl:sort/@select',
+          xsltDocument,
+          null,
+          XPathResult.ORDERED_NODE_ITERATOR_TYPE,
+        )
+        .iterateNext();
+      expect(sort?.nodeValue).toEqual('Title');
+
+      const withParam = xsltDocument
+        .evaluate(
+          '/xsl:stylesheet/xsl:template/ShipOrder/xsl:apply-templates/xsl:with-param[`@name`="prefix"]/@select',
+          xsltDocument,
+          null,
+          XPathResult.ORDERED_NODE_ITERATOR_TYPE,
+        )
+        .iterateNext();
+      expect(withParam?.nodeValue).toEqual("'Item:'");
   });
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/ui/src/services/mapping-serializer.service.test.ts` around lines 405
- 420, The test 'should round-trip UnknownMappingItem verbatim' currently only
asserts the `@select` attribute value; instead update the test to assert that the
entire unknown subtree/fragment is preserved after deserialize+serialize: parse
the serialized XSLT (as done) and locate the whole xsl:apply-templates element
(or its parent unknown fragment) returned by MappingSerializerService.serialize,
then compare its serialized XML text (or inner XML) against the original unknown
fragment from getUnknownApplyTemplateXslt(); ensure you still use
getUnknownApplyTemplateXslt(), MappingSerializerService.deserialize and
MappingSerializerService.serialize and compare the full node content (not just
applyTemplates?.nodeValue) to confirm subtree preservation.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/ui/src/services/visualization.service.ts`:
- Around line 179-183: The generatePrimitiveDocumentChildren routine creates
child nodes using new MappingNodeData directly, which bypasses the
unknown-mapping handling added in createNodeDataFromMappingItem; update
generatePrimitiveDocumentChildren to call createNodeDataFromMappingItem(parent,
mapping) for each mapping so FieldItem, UnknownMappingItem and other MappingItem
types produce the correct FieldItemNodeData, UnknownMappingNodeData or
MappingNodeData instances (ensuring it uses the existing
createNodeDataFromMappingItem helper instead of constructing MappingNodeData
directly).

---

Outside diff comments:
In `@packages/ui/src/services/mapping-serializer.service.ts`:
- Around line 146-152: The global sort
(MappingSerializerService.sortMappingItem) is reordering UnknownMappingItem
instances introduced by MappingSerializerService.populateUnknownMappingItem,
violating the "preserve and write back as-is" requirement; update the sort
comparator (sortMappingItem) to detect when either operand is an
UnknownMappingItem (or both) and return 0 to preserve original sibling order for
unknown items (i.e., do not change relative order of UnknownMappingItem vs
siblings), then continue calling MappingSerializerService.populateMapping as
before for mapping.children so unknown items are written back unchanged.

---

Nitpick comments:
In `@packages/ui/src/services/mapping-serializer.service.test.ts`:
- Around line 405-420: The test 'should round-trip UnknownMappingItem verbatim'
currently only asserts the `@select` attribute value; instead update the test to
assert that the entire unknown subtree/fragment is preserved after
deserialize+serialize: parse the serialized XSLT (as done) and locate the whole
xsl:apply-templates element (or its parent unknown fragment) returned by
MappingSerializerService.serialize, then compare its serialized XML text (or
inner XML) against the original unknown fragment from
getUnknownApplyTemplateXslt(); ensure you still use
getUnknownApplyTemplateXslt(), MappingSerializerService.deserialize and
MappingSerializerService.serialize and compare the full node content (not just
applyTemplates?.nodeValue) to confirm subtree preservation.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 17bd3e7c-9340-44ea-b83d-9a6a369c9bb4

📥 Commits

Reviewing files that changed from the base of the PR and between 4bdab51 and b82b3d5.

📒 Files selected for processing (11)
  • packages/ui/src/components/Document/NodeTitle.test.tsx
  • packages/ui/src/components/Document/NodeTitle.tsx
  • packages/ui/src/components/Document/TargetDocumentNode.tsx
  • packages/ui/src/models/datamapper/mapping.ts
  • packages/ui/src/models/datamapper/visualization.ts
  • packages/ui/src/services/mapping-serializer.service.test.ts
  • packages/ui/src/services/mapping-serializer.service.ts
  • packages/ui/src/services/visualization.service.test.ts
  • packages/ui/src/services/visualization.service.ts
  • packages/ui/src/stubs/datamapper/data-mapper.ts
  • packages/ui/src/stubs/datamapper/xml/UnknownApplyTemplate.xsl

Comment thread packages/ui/src/services/visualization.service.ts
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 17, 2026

Codecov Report

❌ Patch coverage is 98.30508% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 90.04%. Comparing base (9006db9) to head (d57eed1).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
packages/ui/src/services/visualization.service.ts 94.11% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3042      +/-   ##
==========================================
+ Coverage   90.01%   90.04%   +0.03%     
==========================================
  Files         564      565       +1     
  Lines       21062    21105      +43     
  Branches     4772     4782      +10     
==========================================
+ Hits        18958    19004      +46     
+ Misses       2102     2099       -3     
  Partials        2        2              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread packages/ui/src/components/Document/NodeTitle.tsx Outdated
@igarashitm
Copy link
Copy Markdown
Member Author

Also added a header in the popover, thanks for the idea @PVinaches 👍
Screenshot From 2026-03-18 09-00-54

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
packages/ui/src/services/mapping-serializer.service.test.ts (1)

410-452: Assert the full unknown subtree, not just the root attribute.

The preservation contract here is the DOM fragment. These assertions still pass if nested unsupported children like xsl:sort or xsl:with-param are dropped, and the “mixed children” case only covers the already-sorted field-then-unknown order. Please compare the serialized xsl:apply-templates subtree against the original fragment and add one reversed-order case.

Based on learnings SortItem is declared as a model object only and is not yet supported. Full support is planned for a future subtask.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/ui/src/services/mapping-serializer.service.test.ts` around lines 410
- 452, Update the two tests so they assert the entire unknown
xsl:apply-templates subtree is preserved (not just the `@select` attribute) and
add a reversed-order mixed-children case where apply-templates appears before
xsl:attribute to ensure order handling both ways; specifically, in the "should
round-trip UnknownMappingItem verbatim" test (which uses
getUnknownApplyTemplateXslt and MappingSerializerService.deserialize/serialize)
compare the serialized xsl:apply-templates element's full XML/text content
against the original fragment, and in the "should sort UnknownMappingItem after
FieldItems when serializing mixed children" test (which builds mixedXslt and
calls MappingSerializerService.deserialize/serialize) add a new case with the
children reversed and assert that both the entire apply-templates subtree and
the final children ordering match expectations.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/ui/src/components/Document/NodeTitle/UnknownMappingLabel.tsx`:
- Around line 15-29: The Label inside UnknownMappingLabel.tsx is not
keyboard-accessible so the Popover can't be opened by keyboard; make the Label
interactive by adding isClickable and an onClick that toggles/opens the Popover
(or wrap the Label in a semantic <button>), ensuring the Popover
(triggerAction="click") receives keyboard activation via Enter/Space; update the
Label props (the Label element that currently uses color="yellow"
icon={<WarningTriangleIcon />} style={{ cursor: 'pointer' }}) to include
isClickable and a handler that opens the preview so Tab + Enter/Space works for
keyboard users.

In `@packages/ui/src/services/visualization.service.ts`:
- Around line 103-112: The current predicate assigned to
filterPriorityMappingItem replaces UnknownMappingItem with ValueSelector when
parent.isPrimitive, causing unknown mappings to be dropped; update the predicate
logic in the block that builds mappings (the filter using
filterPriorityMappingItem) so that UnknownMappingItem instances are always
included and ValueSelector is included when parent.isPrimitive (e.g., predicate
returns m instanceof UnknownMappingItem || (parent.isPrimitive && m instanceof
ValueSelector)), then continue to call
VisualizationService.createNodeDataFromMappingItem(parent as TargetNodeData, m)
for each filtered mapping.

---

Nitpick comments:
In `@packages/ui/src/services/mapping-serializer.service.test.ts`:
- Around line 410-452: Update the two tests so they assert the entire unknown
xsl:apply-templates subtree is preserved (not just the `@select` attribute) and
add a reversed-order mixed-children case where apply-templates appears before
xsl:attribute to ensure order handling both ways; specifically, in the "should
round-trip UnknownMappingItem verbatim" test (which uses
getUnknownApplyTemplateXslt and MappingSerializerService.deserialize/serialize)
compare the serialized xsl:apply-templates element's full XML/text content
against the original fragment, and in the "should sort UnknownMappingItem after
FieldItems when serializing mixed children" test (which builds mixedXslt and
calls MappingSerializerService.deserialize/serialize) add a new case with the
children reversed and assert that both the entire apply-templates subtree and
the final children ordering match expectations.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 68e2c808-460d-479f-a0cb-d8b4fe3a81ee

📥 Commits

Reviewing files that changed from the base of the PR and between 5009064 and 8ec8023.

📒 Files selected for processing (15)
  • packages/ui/src/components/Document/AddMappingNode.tsx
  • packages/ui/src/components/Document/NodeTitle/NodeTitle.scss
  • packages/ui/src/components/Document/NodeTitle/NodeTitle.test.tsx
  • packages/ui/src/components/Document/NodeTitle/NodeTitle.tsx
  • packages/ui/src/components/Document/NodeTitle/UnknownMappingLabel.tsx
  • packages/ui/src/components/Document/SourceDocumentNode.tsx
  • packages/ui/src/components/Document/TargetDocumentNode.tsx
  • packages/ui/src/models/datamapper/mapping.ts
  • packages/ui/src/models/datamapper/visualization.ts
  • packages/ui/src/services/mapping-serializer.service.test.ts
  • packages/ui/src/services/mapping-serializer.service.ts
  • packages/ui/src/services/visualization.service.test.ts
  • packages/ui/src/services/visualization.service.ts
  • packages/ui/src/stubs/datamapper/data-mapper.ts
  • packages/ui/src/stubs/datamapper/xml/UnknownApplyTemplate.xsl
🚧 Files skipped from review as they are similar to previous changes (3)
  • packages/ui/src/models/datamapper/visualization.ts
  • packages/ui/src/components/Document/TargetDocumentNode.tsx
  • packages/ui/src/stubs/datamapper/xml/UnknownApplyTemplate.xsl

Comment thread packages/ui/src/services/visualization.service.ts
lordrip
lordrip previously approved these changes Mar 18, 2026
Copy link
Copy Markdown
Member

@lordrip lordrip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, @igarashitm, please confirm if the second coderabbit's issue is valid, just in case

Fixes: KaotoIO#2716

When deserializing XSLT, preserve XSLT control elements that is not recognized by DataMapper as an unknown item, keep the DOM fragment as it is so that it could be written back when DataMapper generate the XSLT on serialization
@igarashitm igarashitm requested a review from a team March 18, 2026 14:42
@sonarqubecloud
Copy link
Copy Markdown

@igarashitm
Copy link
Copy Markdown
Member Author

Thank you for the review @lordrip , fixed the coderabbitai issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DataMapper: Introduce UnknownMappingItem/UnknownMappingNodeData and display unrecognized item

3 participants