-
Notifications
You must be signed in to change notification settings - Fork 36
feat: add translation script and GitHub Action for mobile i18n autofill #108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
markrizkalla
wants to merge
9
commits into
openMF:dev
Choose a base branch
from
markrizkalla:feature/auto-localization-on-pull-request
base: dev
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
fd65a26
feat: add translation script and GitHub Action for mobile i18n autofill
markrizkalla 63eb15b
feat: update snapshot path logic and license reference
markrizkalla e6093fc
Replace GITHUB_TOKEN with PAT_TOKEN in workflow
markrizkalla e6e8a00
Modify mobile i18n autofill workflow
markrizkalla 2c484d7
Change PR trigger to pull_request
markrizkalla eab5c7c
Update GitHub Actions workflow for mobile i18n
markrizkalla 4dee8cc
feat: add support for string-array and plurals resources
markrizkalla a33d7f1
feat(translate): add orphaned translation cleanup and improve XML for…
markrizkalla dca68b2
Refactor snapshot saving logic in translate.py
markrizkalla File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| name: Mobile i18n Autofill (bot PR) | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| locales: | ||
| description: 'Comma-separated locale codes to translate' | ||
| required: false | ||
| default: 'ar,bn,de,en,es,fa,fr,gu,hi,hu,id,km,kn,ml,mr,ms,my,pl,pt,ru,si,sw,te,ur' | ||
| type: string | ||
| model: | ||
| description: 'Gemini model to use for translation' | ||
| required: false | ||
| default: 'gemma-3-27b-it' | ||
| type: string | ||
| batch-size: | ||
| description: 'Number of strings per translation batch' | ||
| required: false | ||
| default: '15' | ||
| type: string | ||
| repo-root: | ||
| description: 'Module root to scope translation (e.g. ./feature/settings)' | ||
| required: false | ||
| default: '.' | ||
| type: string | ||
|
|
||
| pull_request: | ||
| types: [labeled] | ||
| branches: [dev] | ||
|
|
||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| i18n-autofill: | ||
| if: >- | ||
| github.event_name == 'workflow_dispatch' || | ||
| github.event.label.name == 'needs-translation' | ||
| runs-on: ubuntu-latest | ||
|
|
||
| env: | ||
| LOCALES: ${{ inputs.locales || 'ar,bn,de,en,es,fa,fr,gu,hi,hu,id,km,kn,ml,mr,ms,my,pl,pt,ru,si,sw,te,ur' }} | ||
| MODEL: ${{ inputs.model || 'gemma-3-27b-it' }} | ||
| BATCH_SIZE: ${{ inputs.batch-size || '15' }} | ||
| REPO_ROOT: ${{ inputs.repo-root || '.' }} | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
| repository: ${{ github.event.pull_request.head.repo.full_name || github.repository }} | ||
| ref: ${{ github.event.pull_request.head.ref || github.ref }} | ||
| token: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Set up JDK 21 | ||
| uses: actions/setup-java@v4 | ||
| with: | ||
| java-version: '21' | ||
| distribution: 'temurin' | ||
|
|
||
| - name: Set up Python 3.11 | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.11' | ||
|
|
||
| - name: Install Python dependencies | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install google-genai lxml | ||
|
|
||
| - name: Run translation autofill | ||
| env: | ||
| GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }} | ||
| run: | | ||
| python translate.py \ | ||
| --mode apply \ | ||
| --repo-root "$REPO_ROOT" \ | ||
| --locales "$LOCALES" \ | ||
| --model "$MODEL" \ | ||
| --batch-size $BATCH_SIZE | ||
|
|
||
| - name: Validate Android resources compile | ||
| run: | | ||
| ./gradlew :cmp-android:processDemoDebugResources | ||
|
|
||
| - name: Commit and Push changes | ||
| run: | | ||
| git config user.name "github-actions[bot]" | ||
| git config user.email "github-actions[bot]@users.noreply.github.com" | ||
|
|
||
| git add cmp-android/ cmp-navigation/ feature/ | ||
|
|
||
| if ! git diff --cached --quiet; then | ||
| git commit -m "chore: auto-generate mobile i18n translations" | ||
| git push origin HEAD:${{ github.event.pull_request.head.ref || github.ref_name }} | ||
| else | ||
| echo "No changes to commit." | ||
| fi | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| # Android String Resource Translator (`translate.py`) | ||
|
|
||
| A production-ready Python script for translating Android string resources (`strings.xml` and `arrays.xml`) using the Google Gemini API. | ||
|
|
||
| ## Features | ||
|
|
||
| - **Format Preservation**: Ensures comments, spacing (blank lines), and structure match the source file exactly. | ||
| - **Placeholder & Markup Safety**: Freezes placeholders (e.g., `%s`, `%1$d`) and markup tags (e.g., `<b>`, `<xliff:g>`) before translating to guarantee they are preserved and kept in the correct order. | ||
| - **Source Attribute Propagation**: Copies attributes like `formatted`, `product`, and `tools:*` to the translated strings. | ||
| - **Robust Error Handling**: Includes batch translation with individual string fallback on failure, and automatic retry mechanisms for rate limits (429) or model overloads (503). | ||
| - **Change Detection**: Tracks source strings through a simple hash-based snapshot mechanism (`.translation_snapshots/`). Only new strings and strings whose source text modified are re-translated, saving time and tokens. | ||
| - **Advanced Resource Support**: Translates single `<string>`, ordered `<string-array>`, and `<plurals>` resources out-of-the-box. | ||
| - **Character Compatibility**: Manages HTML entity conversions and robust Android special character escaping. | ||
| - **AAPT2 Compatibility**: Implements proper `xliff` namespace handling to prevent build errors. | ||
|
|
||
| ## Requirements | ||
|
|
||
| 1. Python 3.8+ | ||
| 2. Required packages: | ||
| ```bash | ||
| pip install google-genai lxml | ||
| ``` | ||
| 3. A Google Gemini API Key | ||
|
|
||
| ## Usage | ||
|
|
||
| Set your Google Gemini API key as an environment variable: | ||
| ```bash | ||
| export GEMINI_API_KEY=your_api_key_here | ||
| ``` | ||
| *(You can customize the environment variable name via the `--api-key-env` flag).* | ||
|
|
||
| ### Applying Translations | ||
|
|
||
| Run the script in `apply` mode to fetch missing strings and write translated files directly to their respective `values-{locale}` folders. | ||
|
|
||
| ```bash | ||
| # Basic usage | ||
| python translate.py --mode apply --locales es,de,fr | ||
|
|
||
| # Using a specific model and fine-tuning batch parameters | ||
| python translate.py \ | ||
| --mode apply \ | ||
| --repo-root . \ | ||
| --locales ar \ | ||
| --model gemma-3-27b-it \ | ||
| --batch-size 15 \ | ||
| --request-delay 4.0 | ||
| ``` | ||
|
|
||
| ### Checking for Missing Translations | ||
|
|
||
| Run the script in `check` mode inside CI/CD workflows to simply verify whether all strings are translated without making any actual API calls or file modifications. | ||
|
|
||
| ```bash | ||
| python translate.py --mode check --locales es,de,fr | ||
| ``` | ||
| *In `check` mode, the script exits with code `2` if translations are missing.* | ||
|
|
||
| ## Available Command-Line Arguments | ||
|
|
||
| - `--mode` (Required): `apply` (to translate and write xml) or `check` (to only check for missing keys). | ||
| - `--locales`: A comma-separated list of target Android language/region codes (e.g. `es,fr,de,ar`). Default is `es,de`. | ||
| - `--repo-root`: The path to the root of the Android project (where to search for `src/*/res/values/strings.xml` or Compose Multiplatform equivalent). Default is `.`. | ||
| - `--model`: The Gemini API model to use. Default is `gemini-2.0-flash`. | ||
| - `--batch-size`: Number of strings to send in a single Gemini API request. Default is `20` (capped at `15` for Gemma models). | ||
| - `--request-delay`: Delay in seconds between API requests to prevent immediate rate-limiting. Default is `2.0` (forced to `4.0` for Gemma models). | ||
| - `--api-key-env`: Name of the environment variable used to retrieve the API key. Default is `GEMINI_API_KEY`. | ||
| - `--no-validate`: Disable automatic malformed XML checks after writing translations. | ||
| - `--verbose` / `-v`: Enable debug-level logging. | ||
|
|
||
| ## Under The Hood | ||
|
|
||
| ### 1. Snapshot Tracking | ||
| When you successfully translate strings, the script saves a JSON file in `.translation_snapshots/` within the source module. Subsequent runs will compare current source text against these hashes, allowing `translate.py` to seamlessly fix previously translated strings if you tweak the original English wording. | ||
|
|
||
| ### 2. Orphaned Translations cleanup | ||
| In `apply` mode, if a developer deletes a string or an array item from the english source, the script reliably detects and strips the orphaned translation from all localized strings files to avoid accumulation of unused strings. | ||
|
|
||
| ### 3. Rate Limit Handling | ||
| If the Google Gemini backend responds with `429 Rate limited` or `503 Service Unavailable`, `translate.py` will automatically backoff and retry according to `--max-retries` and the wait times embedded in API responses. | ||
markrizkalla marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Workflow violates repository standard for reusable workflows.
This job is implemented inline, but workflow files in this repo must delegate to reusable workflows from
openMF/mifos-x-actionhub@v1.0.8.As per coding guidelines
.github/workflows/*.{yml,yaml}: Configure GitHub Actions workflows to use reusable workflows from openMF/mifos-x-actionhub@v1.0.8.🧰 Tools
🪛 actionlint (1.7.11)
[error] 89-89: "github.event.pull_request.head.ref" is potentially untrusted. avoid using it directly in inline scripts. instead, pass it through an environment variable. see https://docs.github.com/en/actions/reference/security/secure-use#good-practices-for-mitigating-script-injection-attacks for more details
(expression)
🤖 Prompt for AI Agents