Skip to content

Use grep -Po for tag_name and browser_download_url extraction globally; cache API responses for arch-specific checks#8016

Merged
MichaIng merged 7 commits intodev-StS_UptimeKumaInstallationfrom
copilot/sub-pr-8013
Mar 14, 2026
Merged

Use grep -Po for tag_name and browser_download_url extraction globally; cache API responses for arch-specific checks#8016
MichaIng merged 7 commits intodev-StS_UptimeKumaInstallationfrom
copilot/sub-pr-8013

Conversation

Copy link
Contributor

Copilot AI commented Mar 14, 2026

The mawk-based JSON field extraction (/^ *"field": "[^"]*",$/{print $4}) fails silently when GitHub returns single-line JSON, causing version detection to fall back to hardcoded values. grep -Po with PCRE \K works on both single- and multi-line responses.

dietpi/dietpi-software

  • Replace all mawk tag_name patterns with grep -Po '"tag_name": *"\K[^"]+' (FreshRSS, ruTorrent, Box86, Box64, Immich ×2; Uptime Kuma was already fixed)
  • Replace all mawk -F\" browser_download_url patterns with grep -Po equivalents (41 patterns across all software entries)

.github/workflows/update_urls.bash

tag_name pattern fix (7 entries):

# Before
aCHECK[$software_id]='curl -sSf '\''https://api.github.com/.../releases/latest'\'' | mawk -F\" '\''/^ *"tag_name": "[^"]*",$/{print $4}'\'
# After
aCHECK[$software_id]='curl -sSf '\''https://api.github.com/.../releases/latest'\'' | grep -Po '\''"tag_name": *"\K[^"]+'\'

browser_download_url pattern fix (39 entries):

# Before (double-quoted, with arch vars)
aCHECK[$software_id]='echo "$response" | mawk -F\" "/^ *\"browser_download_url\": \".*PATTERN\"$/{print \$4}"'
# After
aCHECK[$software_id]='echo "$response" | grep -Po "\"browser_download_url\": *\"\K[^\"]*PATTERN"'

# Before (single-quoted, static pattern, with '\'' quoting in aCHECK)
aCHECK[$software_id]='curl -sSf '\''URL'\'' | mawk -F\" '\''/^ *"browser_download_url": ".*PATTERN"$/{print $4}'\'
# After
aCHECK[$software_id]='curl -sSf '\''URL'\'' | grep -Po '\''"browser_download_url": *"\K[^"]*PATTERN'\'

Special cases: the Forgejo mawk -v RS=, pattern (for single-line JSON) is replaced with grep -Po "..." | head -1 to replicate the original {exit} first-match behaviour.

Response caching for all 32 aARCH entries — avoids N redundant curl calls (one per architecture) per software ID:

  • Add aURL[$software_id]='https://...' for every entry with aARCH defined
  • Replace curl 'URL' | with echo "$response" | in aCHECK
  • Loop fetches once before the arch iteration:
response=
if [[ ${aURL[i]} ]]
then
    if [[ ${GH_TOKEN} && ${aURL[i]} == 'https://api.github.com/'* ]]
    then
        response=$(curl -sSf -H "Authorization: token $GH_TOKEN" "${aURL[i]}")
    else
        response=$(curl -sSf "${aURL[i]}")
    fi
fi
  • The existing GitHub token string-substitution in aCHECK is retained with a comment clarifying it only applies to entries without aURL

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

…RCH entries

Co-authored-by: MichaIng <28480705+MichaIng@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix Uptime Kuma installation to use the latest version Use grep -Po for tag_name extraction globally; cache API responses for arch-specific checks Mar 14, 2026
Copilot AI requested a review from MichaIng March 14, 2026 20:48
@MichaIng
Copy link
Owner

@copilot
Please apply the mawk => grep -Po migration as well to those GitHub API calls where we match browser_download_url.

…software and update_urls.bash

Co-authored-by: MichaIng <28480705+MichaIng@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 14, 2026

@copilot
Please apply the mawk => grep -Po migration as well to those GitHub API calls where we match browser_download_url.

Done in commit dc7808b. All mawk -F\" patterns for browser_download_url have been migrated to grep -Po in both files:

  • dietpi/dietpi-software: 41 patterns converted (31 double-quoted with $arch/other vars, 8 single-quoted static, 1 EasyRSA special without ^ * prefix, 1 Forgejo RS=, with | head -1 added to replicate mawk's {exit} behavior)
  • .github/workflows/update_urls.bash: 39 patterns converted (30 double-quoted using echo "$response", 8 single-quoted with '\'' quoting in aCHECK assignments, 1 Forgejo RS=, with | head -1)

Copilot AI changed the title Use grep -Po for tag_name extraction globally; cache API responses for arch-specific checks Use grep -Po for tag_name and browser_download_url extraction globally; cache API responses for arch-specific checks Mar 14, 2026
Since we do not want to match RC versions, use lookahead in pattern to match the double-quotation closing.
@MichaIng MichaIng marked this pull request as ready for review March 14, 2026 22:20
@MichaIng MichaIng merged commit eab78d2 into dev-StS_UptimeKumaInstallation Mar 14, 2026
2 checks passed
@MichaIng MichaIng deleted the copilot/sub-pr-8013 branch March 14, 2026 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GitHub API parsing sometimes fails due to minified JSON response

2 participants