Repository navigation
[#1156] Publish wiki How-To guides - #1218
Conversation
maximthomas
left a comment
There was a problem hiding this comment.
praise: The migration's outputs check out where they can be measured.
wiki-how-to.patchpassesgit apply --checkat wikic68baf2f, which is still the live wiki HEAD, and each of its 55 distincthttps://doc.openidentityplatform.org/openam/how-to/<slug>targets has a page in the Antora output (target/asciidoc/antora/modules/how-to/pages, local build at this head).- All 33 hard-coded reference fragments in
how-to/*.adocresolve, including the generatedchap-auth-modules#adaptive-risk--module-ref. - The
build-maven (ubuntu-latest, 11)leg convertspdf-source/how-to/index.adocunderMAVEN_OPTS=-Xmx4g, andHowTo_Guides.pdfcomes out at 28.0 MB, well under GitHub's 100 MB file limit for the wiki publish step.
issue (non-blocking): The javac classpath names three OpenSSO-era jars that no build produces, and the jar list omits openam-shared.
openam-documentation/openam-doc-source/src/main/asciidoc/how-to/how-to-develop-a-custom-authentication-module.adoc:502, :45
The OpenAM WAR ships no amserver.jar, opensso-sharedlib.jar or opensso.jar. The listing also imports com.sun.identity.shared.*, which is in openam-shared. Compiled with JDK 17 against the 16.2.0-SNAPSHOT jars, the page's two listings build against openam-core + openam-shared. Against openam-core alone, as line 45 says, they fail with package com.sun.identity.shared.debug does not exist. Servlet API was not needed. Other JDKs were not tried.
* openam-core++*++.jar
* openam-shared++*++.jar$ javac -d ../classes/ -classpath 'lib/*' SampleAuth.java SampleAuthPrincipal.javasuggestion (non-blocking): Nothing checks the 33 hard-coded doc.openidentityplatform.org/openam/...#<id> fragments.
openam-documentation/openam-doc-source/src/main/asciidoc/how-to/openam-vs-keycloak.adoc:64-75
Neither antora nor asciidoc-to-pdf resolves link fragments. 24 of these links point at reference/chap-auth-modules, whose ids openam-doc-services-ref generates from i18n strings. For example, adaptive-risk--module-ref gets its double hyphen from the trailing space in amAuthAdaptive.properties:30. Renaming a link to #adaptive-risk-module-ref, or trimming that space, stays green in every build. The check below passes at this head and fails on that one-character mutant (broken anchor: …#adaptive-risk-module-ref).
# after the doc build, from openam-documentation/openam-doc-source
grep -rhoE 'doc\.openidentityplatform\.org/openam/[a-z-]+/[a-z-]+#[A-Za-z0-9_.-]+' src/main/asciidoc/how-to \
| sort -u | while read -r u; do
r=${u#doc.openidentityplatform.org/openam/}; page=${r%%#*}; id=${r#*#}
grep -qE "(\[\[$id(,[^]]*)?\]\]|\[#$id([.,%][^]]*)?\])" "target/asciidoc/source/$page.adoc" \
|| { echo "broken anchor: $u"; exit 1; }
donePin: the step above in build.yml after the Maven build. It does not cover xref: anchors or the how-to pages' own anchors.
issue (non-blocking): The first install step's heading is published as the literal text ###Copy the jar file.
openam-documentation/openam-doc-source/src/main/asciidoc/how-to/how-to-develop-a-custom-authentication-module.adoc:516
The wiki line had no space after the hashes, so it converted to ++###++Copy the jar file. Its three siblings became anchors plus ==== headings (lines 525, 535, 545). In HTML and PDF the step has no heading, TOC entry or anchor. No other ++###++ remains under how-to/.
[[how-to-develop-a-custom-authentication-module-copy-the-jar-file]]
==== Copy the jar filequestion (non-blocking): Was keeping the wiki listings deliberate, given that line 24 presents the in-repo sample as the page's source code?
openam-documentation/openam-doc-source/src/main/asciidoc/how-to/how-to-develop-a-custom-authentication-module.adoc:24, :158, :186, :553
The page uses package com.forgerock.openam.examples, user test and attribute sampleauth-service-specific-attribute. openam-samples/custom-authentication-module uses org.forgerock.openam.examples (SampleAuth.java:25), user demo (:65) and specificAttribute (amAuthSampleAuth.xml:44). A reader who builds the linked sample and then runs line 553's register-auth-module … -a com.forgerock.openam.examples.SampleAuth registers a class that is not in that jar. The page is consistent with itself, so this is minor either way. If the mismatch is not deliberate, say on line 24 that the listings are a simplified variant (sample class org.forgerock.openam.examples.SampleAuth, user demo), or align the listings with the sample.
suggestion (non-blocking): The wiki stubs drop every section anchor, so deep links into the moved articles land at the top of the page.
openam-documentation/wiki-migration/wiki-how-to.patch (all 54 stubs, e.g. the Passwordless-Authenticaion-Methods stub near line 6456)
Each stub has three lines and one link without a fragment. An inbound link such as wiki/Passwordless-Authenticaion-Methods#conclusion used to land on the section. After the patch it lands on the stub, and from there on the top of the new page, even though the new pages keep their section anchors. Keeping each article's headings in its stub, each followed by a link to the matching anchor on the new page, would keep those deep links working.
suggestion (non-blocking): The articles mix relative xrefs with hard-coded, unversioned links to the same site.
openam-documentation/openam-doc-source/src/main/asciidoc/how-to/openam-vs-keycloak.adoc:64-75 and the other doc.openidentityplatform.org/openam/... links under how-to/
19 links use xref:../reference/openam-cli-tools.adoc, and about 33 use https://doc.openidentityplatform.org/openam/<guide>/<chapter>#<id>. No other guide hard-codes a self-site URL. The site serves master at /openam/ and older lines from release tags (antora-playbook.yml: OpenAM-15.2.2). Once this line becomes an older version, the absolute links on its how-to pages will point at the newer reference, while its xrefs stay within the same version. Converting them to xref:../<guide>/<chapter>.adoc#<id>[…] keeps them in version. modules/reference/pages/chap-auth-modules.adoc exists in the Antora output. The check above would then have to match the xref form. I did not trace how asciidoc-to-pdf rewrites xrefs in the PDF.
nitpick (non-blocking): The text says "properties file" under the heading for the XML file.
openam-documentation/openam-doc-source/src/main/asciidoc/how-to/how-to-develop-a-custom-authentication-module.adoc:538
We copy the authentication module XML file into the config/auth/default directory within the openam web application deployed into Tomcat.nitpick (non-blocking): Two wiki leftovers are published as text: a dangling "or here." and the Confluence emoticon "(wink)".
openam-documentation/openam-doc-source/src/main/asciidoc/how-to/how-to-develop-a-custom-authentication-module.adoc:47, :475
Line 47 should end "…from within your favourite container." Line 475 should read "(There is also a second naming convention; please refer to the source code.)".
nitpick (non-blocking): The patch rewrites three Home.md entries but keeps their misspelled labels, although its own stubs use the corrected titles.
openam-documentation/wiki-migration/wiki-how-to.patch:695, :732, :740
"Authenticaion", "Contaners" and "Devleop" should be "Authentication", "Containers" and "Develop" on those three added lines.
|
Addressed the review feedback in ece2f1c78f:
Validation: AsciiDoc/Antora generation; 55 rendered How-To pages checked for local links, fragments and images (232 image occurrences), with no How-To diagnostics or unresolved elements; the 318-page PDF rebuilt and its Adaptive Risk link annotation and restored heading checked; deferred wiki patch passes The two remaining navigation follow-ups are tracked in #1220: preserving incoming wiki section links before applying the deferred patch, and adding version-specific cross-guide PDF URLs in the shared documentation converter. The wiki patch remains deferred until the documentation and companion site navigation have deployed and been verified. |
maximthomas
left a comment
There was a problem hiding this comment.
praise: The round-2 link work holds up when it is built and run.
openam-documentation/check-how-to-links.py, run over this head's generated source (a localopenam-doc-sourcebuild), reportsChecked 136 How-To xrefs in 55 pages; 0 errors.. It still rejects a one-character anchor typo.- The antora goal rewrites every new version-aware cross-guide xref.
target/asciidoc/antora/modules/how-to/pageshas noxref:../left, and the 60 cross-guide links becomereference:(44),admin-guide:(5) and other page ids. - All the round-1 text fixes are in:
==== Copy the jar file, theopenam-core/openam-sharedclasspath, no(wink)oror here., and no hard-codeddoc.openidentityplatform.org/openam/URLs left inhow-to/*.adoc.
issue (non-blocking): The compile and package listings work in two different directories, so following them in order fails at cd classes.
openam-documentation/openam-doc-source/src/main/asciidoc/how-to/how-to-develop-a-custom-authentication-module.adoc:505, :512-513, :526, :536, :546
javac -d ../classes/ writes the classes next to the working directory, not inside it. On JDK 11 and later it creates that directory itself, so the compile succeeds. The next listing then runs cd classes, which fails with "no such file or directory", and no jar is built. Changing it to cd ../classes alone would leave the three cp commands (:526, :536, :546) in the wrong directory. The -d ../classes/ line came from the round-1 review's suggested fix, which left the next listing unchanged. The fix below keeps every step in the directory that holds the sources and lib/. :526, :536 and :546 then run unchanged, and :556 already takes a full path. I ran the two commands on JDK 11.0.25, 17.0.11 and Zulu 26: the jar contains com/forgerock/openam/examples/*.class and is written next to the sources. Not checked: the page never says where amAuthSampleAuth.properties and SampleAuth.xml are created. The fix assumes they are next to the sources.
[source,bash]
----
$ javac -d classes -classpath 'lib/*' SampleAuth.java SampleAuthPrincipal.java
----
This will create our class file that we need to install into OpenAM. Next we should package the classes files up into a jar file.
[source,bash]
----
$ jar cf sampleauth.jar -C classes .
----issue (non-blocking): The Home.md header in the wiki patch declares a post-image blob that the patch no longer produces.
openam-documentation/wiki-migration/wiki-how-to.patch:656
This head edits three added Home.md lines by hand but keeps index a4f95a6..b15e472. Applied to the wiki at c68baf2f, the patch now produces Home.md blob f946b8c. The previous head's patch produced b15e472. git apply and --3way still work because they use only the pre-image, so the only effect is a header that disagrees with the result it produces. The patch has no binary hunks, so a plain git diff regenerates it. Doing that after the label change below fixes both comments at once.
# in the OpenAM.wiki checkout at c68baf2f, with the edits applied
git diff > /path/to/openam-documentation/wiki-migration/wiki-how-to.patchsuggestion (non-blocking): The link checker passes the Adaptive Risk xref whether or not it keeps its ++ passthrough.
openam-documentation/check-how-to-links.py:39-40
The checker strips ++…++ and looks up the bare id. Asciidoctor, however, replaces -- between word characters before it resolves an xref. Without the passthrough, openam-vs-keycloak.adoc:67 would render as #adaptive-risk—​module-ref, a dead link (measured with Asciidoctor 2.0.20). I ran the mutant on this head's generated source: with the ++ removed, the checker still reports 0 errors. With the change below it reports 1 error, and the unmutated source still reports 0. The change counts that xref as not checked (135), because the branch skips the counter.
if anchor.startswith("++") and anchor.endswith("++"):
anchor = anchor[2:-2]
elif "--" in anchor:
# Asciidoctor replaces "--" between word characters with an em dash
# before it resolves the xref, so such a fragment needs ++...++.
errors.append(f"{article.name}: fragment needs ++...++: {target}")
continuePin: remove the ++ at openam-vs-keycloak.adoc:67 in the generated source. The check must report 1 error.
suggestion (non-blocking): The link checker accepts an anchor id that exists only inside a comment or listing block.
openam-documentation/check-how-to-links.py:47-50
The id patterns ^\[\[ and ^\[# run over the raw target file, so an [[id]] inside ////…//// or [#id] inside ----…---- counts as a defined anchor. Asciidoctor registers neither, so the rendered link would be dead while CI stays green. I tested both mutants on this head's generated source: each reports 0 errors now and 1 error with the change below. The unmutated source still reports 0. None of the current 136 xrefs is known to depend on this.
text = destination.read_text(encoding="utf-8")
# Asciidoctor registers no id inside comment, listing or literal blocks.
text = re.sub(r"^(/{4,}|-{4,}|\.{4,})[ \t]*\n.*?^\1[ \t]*$", "",
text, flags=re.MULTILINE | re.DOTALL)Pin: point the Adaptive Risk xref at #zz-bogus and append ////, [[zz-bogus]], //// to reference/chap-auth-modules.adoc. The check must report 1 error.
nitpick (non-blocking): Three Home.md labels in the patch differ from the titles of the stubs and pages they link to.
openam-documentation/wiki-migration/wiki-how-to.patch:697, :740, :744
The same patch's stubs (:13, :2048, :393) and the adoc titles use "Adaptive Authentication: How to Improve…", "Creating a Custom Authentication Module" and "Captcha module".
* [Adaptive Authentication: How to Improve Security Without Annoying Users](https://doc.openidentityplatform.org/openam/how-to/adaptive-authentication-how-to-improve-security-without-annoying-users)
* [Creating a Custom Authentication Module](https://doc.openidentityplatform.org/openam/how-to/how-to-develop-a-custom-authentication-module)
* [Captcha module](https://doc.openidentityplatform.org/openam/how-to/captcha-module)|
Addressed all five round-2 comments in 5cf2497f7d:
Validation: five regression tests pass; the generated-source check still reports 136 xrefs in 55 pages with 0 errors; source whitespace checks pass. The full Maven/PDF and site builds were not rerun for these fixes. The PR description now distinguishes this validation from the earlier migration builds. The wiki patch remains deferred. The navigation follow-ups remain tracked in #1220, including preserving incoming wiki section links before applying the patch. |
maximthomas
left a comment
There was a problem hiding this comment.
praise: The round-2 link-checker gaps are closed the way Asciidoctor actually behaves.
- The verbatim-block strip in
check-how-to-links.py:53agrees with Asciidoctor 2.0.20 on all six probes this review ran: longer delimiters, trailing whitespace, a different delimiter inside a block, and an anchor between two blocks. \w--\wfragments without++now fail with "fragment needs ++...++" (check-how-to-links.py:42), andtest_double_hyphen_between_word_characters_needs_passthroughpins it.- The custom-auth compile listing (
how-to-develop-a-custom-authentication-module.adoc:502-514) now runs from one directory:javac -d classes, thenjar cf sampleauth.jar -C classes ..
issue (blocking): The documentation PDF build runs out of heap under -Xmx4g on the JDK 11 ubuntu leg.
.github/workflows/build.yml:62, .github/workflows/deploy.yml:68, :75, .github/workflows/release.yml:84
At 5cf2497, build-maven (ubuntu-latest, 11) failed in OpenAM Server Documentation Sources with java.lang.OutOfMemoryError: Java heap space at org.jruby.ext.zlib.JZlibInflate. That frame is PNG decoding, and it failed right after "Converted .../pdf-source/how-to/index.adoc" (job 127268386931). Because the build failed, the new "Check How-To documentation links" step was skipped. At 85cfaf0 the same leg converted all 11 guides (job 127170839065), so with the illustrated How-To guide the peak sits right at 4 GB and the result changes from run to run. deploy.yml:46 and release.yml:50 also run JDK 11 on ubuntu-latest with the same -Xmx4g, so after merge the snapshot deploy that publishes the docs, and a release build, can fail the same way. Only Linux builds the PDFs: profile man-pages in openam-doc-source/pom.xml:89-109 activates on os name Linux, so the macOS and Windows legs can keep their current heap.
- name: Build with Maven
env:
MAVEN_OPTS: ${{ matrix.os == 'ubuntu-latest' && '-Xmx6g' || '-Xmx4g' }} -Dhttps.protocols=TLSv1.2 -Dmaven.wagon.httpconnectionManager.ttlSeconds=120 -Dmaven.wagon.http.retryHandler.requestSentEnabled=true -Dmaven.wagon.http.retryHandler.count=10In deploy.yml:68, deploy.yml:75 and release.yml:84, change -Xmx4g to -Xmx6g. This review did not check two things: whether 6 GB is enough on JDK 11, and the runner's RAM (GitHub documents 16 GB for public-repo ubuntu-latest). Done means build-maven (ubuntu-latest, 11) is green twice at the fix head, once on a rerun. Or: shrink the 222 How-To PNGs until 4 GB holds, checked the same way.
suggestion (non-blocking): No test checks that a verbatim block closes only on its own delimiter, or that the strip stops at the first closing delimiter.
openam-documentation/test_check_how_to_links.py:53-62, openam-documentation/check-how-to-links.py:53
Each block row opens and closes with the same 4-character delimiter, and the one after-block row has only one block. So two mutants of :53 pass all 5 tests. The first lets any delimiter close the block (^(?:/{4,}|-{4,}|\.{4,})[ \t]*$ in place of ^\1[ \t]*$). It also gives 0 errors over the 55 built How-To pages. Under it, an [[id]] inside ----- … ---- … ----- counts as defined, and Asciidoctor 2.0.20 does not register it there. The second replaces the lazy .*? with a greedy .*; today only the real-source run catches it, with 13 errors.
def test_block_closes_only_on_its_own_delimiter(self):
for definition in ("-----\n----\n[[hidden]]\ntext\n-----\n",
"----\n....\n[[hidden]]\ntext\n----\n"):
with self.subTest(definition=definition):
result = self.check_link("hidden", definition)
self.assertEqual(1, result.returncode, result.stdout + result.stderr)
self.assertIn("broken anchor", result.stdout)
def test_anchor_between_two_verbatim_blocks_is_a_destination(self):
result = self.check_link("visible", "----\na\n----\n\n[[visible]]\ntext\n\n----\nb\n----\n")
self.assertEqual(0, result.returncode, result.stdout + result.stderr)Pin: both tests pass at the head. The first fails under the any-delimiter mutant and the exact-4 mutant; the second fails under the greedy mutant. Measured as written.
suggestion (non-blocking): No test row uses a delimiter longer than four characters or one with trailing whitespace.
openam-documentation/test_check_how_to_links.py:54-58, openam-documentation/check-how-to-links.py:53
Three mutants pass all 5 tests and give 0 errors over the built pages: {4,} changed to {4}, [ \t]* dropped on both delimiters, and [ \t]* dropped on the closing delimiter only. Under them, an id inside ----- … -----, ---- … ---- or ---- … ----\t counts as defined, and Asciidoctor 2.0.20 registers none of the three.
def test_longer_and_padded_delimiters_open_and_close_blocks(self):
for definition in ("-----\n[[hidden]]\ntext\n-----\n",
"----\n[[hidden]]\ntext\n---- \n",
"---- \n[[hidden]]\ntext\n----\t\n"):
with self.subTest(definition=repr(definition)):
result = self.check_link("hidden", definition)
self.assertEqual(1, result.returncode, result.stdout + result.stderr)
self.assertIn("broken anchor", result.stdout)Pin: passes at the head and fails under all three mutants. Measured as written.
|
Implemented the round-3 changes in 0cc84f8c8d.
The heap change still needs CI confirmation: two successful The wiki patch remains deferred, and the navigation follow-ups remain tracked in #1220. |
maximthomas
left a comment
There was a problem hiding this comment.
praise: The round-3 heap fix and the proposed tests landed as specified.
.github/workflows/build.yml:62raises the heap only where the PDFs are built (matrix.os == 'ubuntu-latest' && '-Xmx6g' || '-Xmx4g'; profileman-pagesis Linux-only), anddeploy.yml:68,:75andrelease.yml:84get the same-Xmx6g. build-maven (ubuntu-latest, 11) is green at 0cc84f8, including "Check How-To documentation links" (job 127412775032).- The PDF build also passes locally under JDK 11.0.25 at
-Xmx6g(11:23,HowTo_Guides.pdf28 MB). On the same machine it runs out of memory at-Xmx4g. test_check_how_to_links.py:65,:73and:77kill the five strip-regex mutants named in round 3.
Adds OpenAM How-To Guides to the site landing page, OpenAM navigation and PDF mapping. Merge after the companion OpenAM documentation change has deployed its generated `how-to` module. Generated pages and images remain owned by the existing OpenAM publication workflow. Validation: current-version Antora integration build with the companion generated module; all 55 How-To pages checked for local links, fragments and images. No new How-To diagnostics; existing warnings elsewhere remain unchanged. Historical release tags were excluded from the integration playbook. Companion PR: OpenIdentityPlatform/OpenAM#1218. Related issue: OpenIdentityPlatform/OpenAM#1156.
Publishes 54 practical wiki articles as a searchable How-To guide with 222 local images. Registers the guide in the documentation build and adds a deferred wiki migration patch. Companion site PR: OpenIdentityPlatform/doc.openidentityplatform.org#42.
The migrated articles preserve code listings, section anchors and links between articles. Own-site documentation links use version-aware Antora xrefs. The custom authentication module tutorial has corrected compilation, packaging and installation instructions and explains how its preserved wiki listings differ from the repository sample.
Linux Maven builds use a 6 GB heap in build, deploy and release workflows to address the JDK 11 PDF build's
OutOfMemoryErrorat the previous head. macOS and Windows builds retain 4 GB. The Linux/JDK 11 build runs link-checker regression tests and validates generated How-To xrefs against explicit destination anchors, including generated service-reference anchors. The checker rejects unprotected intraword double hyphens and excludes anchor-like text in comment, listing and literal blocks.Validation at this head
git diff --checkpasses for this round.-Xmx6gwas stopped before completion because it was taking too long. It provides no completed PDF-build validation. The fix still needs two successfulbuild-maven (ubuntu-latest, 11)runs at this head, including one rerun; the 6 GB heap is not yet proven sufficient on CI.Earlier validation
-Xmx4g(28,009,356 bytes), and its restored heading and Adaptive Risk link annotation were checked. A later Ubuntu/JDK 11 CI build failed with heap exhaustion, so that earlier success does not establish reliability at 4 GB.openam-coreandopenam-sharedon JDK 11, 17, 21 and 27.c68baf2f36df4bb0d3c0b3c7f96a43832b80e4f3. The patch preserves historical whitespace.Publication order and follow-ups
The wiki patch and inventory/application instructions are included for later application. The patch is not applied automatically.
Publish and verify the generated documentation, then merge and deploy the companion site navigation change. Before applying the wiki patch, complete the incoming wiki section-link preservation tracked in #1220. That issue also tracks version-specific cross-guide PDF URLs in the shared converter. The site build and external integration workflows were not rerun for this round.
Fixes #1156