-
Notifications
You must be signed in to change notification settings - Fork 50
feat: Add image compression support and parallel file processing #2717
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Changes from all commits
Commits
Show all changes
98 commits
Select commit
Hold shift + click to select a range
c32b35b
feat(docs): improve sidebar toggle accessibility
google-labs-jules[bot] 4226780
Merge branch 'develop' into palette-sidebar-a11y-7530072050815763256
srod 2bc5114
Merge branch 'develop' into palette-sidebar-a11y-7530072050815763256
srod e1d7fb8
ci: aggregate GitHub releases to prevent spam
srod 84deb55
Merge branch 'develop' into palette-sidebar-a11y-7530072050815763256
srod 513db8d
Merge pull request #2705 from srod/palette-sidebar-a11y-7530072050815…
srod 1b0d21c
docs: rename CLAUDE.md to AGENTS.md for broader AI tool compatibility
srod f89474d
docs: rename CLAUDE.md to AGENTS.md with expanded guidance
srod 94bdace
ci: add permissions for content read access in GitHub Actions workflow
srod bbefd20
feat(core): implement automatic parallel compression for array outputs
srod b87b445
feat(core): validate input array and preserve 1:1 mapping
srod 268bd40
Merge pull request #2713 from srod/feat/auto-parallel-compression-2
srod 3c98739
feat: add image compression support (sharp, svgo, imagemin)
srod 21d6c4c
feat(cli): add support for modern compressors
srod aaa5c61
docs(examples): add image compressor examples
srod 6981030
fix(types): remove 'as any' cast in CLI and improve generics propagation
srod 940a97b
fix(sharp): update sharp dependency and node engine requirement
srod cc7a538
docs(sharp): sync changelog version with package.json (10.0.0)
srod ecf52a4
docs(changelog): sync imagemin and svgo versions with package.json (1…
srod f55ec87
docs: use absolute URL for README logo images
srod f734936
fix: correctly handle Buffer input in all compressors
srod 6599bed
fix(utils): treat SVG as text-based format
srod c5b6c0d
fix(utils): use first input file for array inputs in run.ts
srod fe1704e
fix(no-compress): safe runtime type checking
srod 2cefbcf
fix(imagemin): update types to accept Buffer or Uint8Array
srod e406a9c
fix(imagemin): add lossy option to GifsicleOptions type
srod ca99a22
fix(imagemin): update mozjpeg options in types and implementation
srod 4d4ad28
fix(imagemin): update pngquant options in types
srod 908ad67
fix(utils): return array of buffers for image array input in compress…
srod 5e3a69b
fix(compressors): explicitly reject array content in all compressors
srod 48b9fd7
perf(compressors): move array check to function start for fail-fast v…
srod b1dc6c3
fix(utils): remove redundant image checks and add lightningcss array …
srod 8d0e393
fix(imagemin): clamp quality, effort and optimizationLevel
srod 1007dda
fix(sharp): clamp quality and effort and update documentation
srod bdd3534
docs(imagemin): add alt text to badges for accessibility
srod bb3305b
docs: add missing alt text to badges across all README files
srod dc449a9
test(utils): add unit tests for writeMultipleOutputs edge cases
srod 2f8b3a5
refactor(utils): add overloads to readFile for better type inference
srod 565c74b
refactor(babel-minify): simplify readFile and content normalization
srod 031d860
docs(utils): update determineContent @returns JSDoc for accuracy
srod 80b5f77
fix(utils): prevent mixing image and text files in input array
srod ce0fc2a
feat(imagemin): implement lossless mode for PNG using imagemin-optipng
srod 503995d
chore: update bun.lock after adding imagemin-optipng
srod b54b9cd
refactor(sharp): allow all supported formats in multi-format conversion
srod 264f091
fix(sharp): add error handling and dedicated compressionLevel for PNG
srod a7273b0
refactor(sharp): return empty string for code on single format conver…
srod d0b7e99
fix(imagemin): respect user optimizationLevel in lossless mode
srod a8cb53c
test(sharp): fix clamping test to capture and assert each result
srod 5344a1a
test(sharp): add robust assertions for quality and effort clamping
srod c1d15f1
refactor: consolidate content validation in compressor packages
srod 3fba701
test: add coverage for edge cases and update svgo to use ensureString…
srod 82fe57c
refactor(test): use dynamic cleanup for created files in utils.test.ts
srod 31e70ce
refactor: remove redundant type assertion in ensureStringContent
srod 8e6a391
test: achieve 100% line coverage in utils package
srod e02de41
test(utils): move cleanup registration to the start of the test
srod bb09542
feat(svgo): add error handling to optimize call
srod 2c0ba16
fix(cli): validate type option for CSS-only compressors
srod b87ada7
test(cli): ensure mock cleanup runs regardless of assertion outcome
srod eb2e3bb
test: achieve 100% line coverage in svgo and utils packages
srod 42adc93
build(imagemin): add missing type packages for plugins
srod 4071650
build(imagemin): use official @types/imagemin and cleanup custom types
srod 6cbfcfc
build(imagemin): update package metadata to remove unsupported WebP f…
srod 46d1040
Merge pull request #2701 from srod/feature/image-support
srod 9e8a44c
fix(deps): update dependency sharp to ^0.34.0
renovate[bot] 0c9ec4a
Merge pull request #2714 from srod/renovate/sharp-0.x
srod f1a957e
feat: Parallelize file compression and use async IO
google-labs-jules[bot] 4287a97
feat: Parallelize file compression and use async IO
google-labs-jules[bot] 09b66f7
feat: Parallelize file compression and use async IO
google-labs-jules[bot] 31fc3e3
feat: Parallelize file compression and use async IO
google-labs-jules[bot] 61a32cd
refactor: Use fully async file validation
google-labs-jules[bot] 8fc4fcf
fix: address lint and type errors
google-labs-jules[bot] 44d50ec
feat: Parallelize file compression and use async IO
google-labs-jules[bot] 084e226
test(core): update async tests to expect error on invalid inputs
srod 8ace82b
fix: resolve lint errors and add test coverage for async file validation
srod 6e751ff
refactor(utils): use async file reading for images in compressSingleFile
srod 08559f6
Merge pull request #2706 from srod/bolt-perf-parallel-compression-129…
srod c95564e
Merge origin/main into develop
srod e130109
test(utils): add coverage for edge cases in writeMultipleOutputs
srod d78831d
docs(AGENTS): add Async / Parallel Patterns subsection
srod baae3e2
docs(sharp): fix effort default and range to match implementation
srod d781507
chore: update workflows, docs and add missing license header
srod 054eeaa
refactor(utils): remove redundant file checks in readFile to avoid TO…
srod d57854b
test(utils): fix inconsistent file paths in multi-output tests
srod 162e633
refactor(utils): use explicit Error instance check in readFile
srod 2d9da8b
📝 Add docstrings to `develop`
coderabbitai[bot] 047eb97
Merge pull request #2719 from srod/coderabbitai/docstrings/162e633
srod 64285d6
fix(sharp): use named type import for Sharp instead of default import
srod 0ec3a47
chore: formatting and biome cleanup
srod 97531b7
fix(imagemin): replace deprecated 'binary' encoding with 'latin1'
srod e720cd8
refactor(sharp): use discriminated union to prevent conflicting forma…
srod 70fa38c
refactor(sharp): extract clamp function to module scope
srod f2f338e
feat(sharp): make mozjpeg option configurable for JPEG encoding
srod 1a840c9
refactor(utils): simplify readFileContentAsync by removing redundant …
srod 78bb2ce
fix(utils): use type-safe error handling in isValidFileAsync
srod f56f69f
docs(cli): clarify that --type flag is required for certain compressors
srod 6659389
refactor(utils): remove unnecessary type cast in run function
srod 0a981e0
refactor(utils): extract getFirstInputFile helper for readability
srod 7bf2442
docs(utils): add JSDoc to getFirstInputFile for consistency
srod File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| ## 2025-12-30 - Sidebar Toggle Accessibility | ||
| **Learning:** `aria-pressed` is for toggle buttons (like Bold/Italic), while `aria-expanded` is for disclosing content (like Menus/Sidebars). Screen readers treat them differently. | ||
| **Action:** Always check if a toggle button controls a region (`aria-expanded`) or changes a state (`aria-pressed`). | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| --- | ||
| "@node-minify/sharp": minor | ||
| "@node-minify/svgo": minor | ||
| "@node-minify/imagemin": minor | ||
| "@node-minify/utils": patch | ||
| "@node-minify/types": patch | ||
| "@node-minify/core": patch | ||
| --- | ||
|
|
||
| feat: Add image compression support | ||
|
|
||
| New packages: | ||
| - `@node-minify/sharp`: Convert and compress images to WebP, AVIF, PNG, JPEG using sharp | ||
| - `@node-minify/svgo`: Optimize SVG files using SVGO | ||
| - `@node-minify/imagemin`: Compress PNG, JPEG, GIF images using imagemin | ||
|
|
||
| Core changes: | ||
| - Support for binary (Buffer) content in compressors | ||
| - Multi-format output support (e.g., convert PNG to both WebP and AVIF) | ||
| - New `buffer` and `outputs` fields in CompressorResult type |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| # Bolt's Journal |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,232 @@ | ||
| # AGENTS.md | ||
|
|
||
| This file provides guidance to AI coding assistants when working with code in this repository. | ||
|
|
||
| ## Requirements | ||
| - **Node.js**: >=20.0.0 | ||
| - **Bun**: 1.3.5+ | ||
|
|
||
| ## Package Manager | ||
|
|
||
| This branch (`main`) uses **Bun** as the package manager and runtime. | ||
|
|
||
| ## Commands | ||
| - **Install**: `bun install` | ||
| - **Build**: `bun run build` | ||
| - **Lint**: `bun run lint` | **Format**: `bun run format` | ||
| - **Test all**: `bun run test` | ||
| - **Test single package**: `bun run test packages/<name>` (e.g., `bun run test packages/core`) | ||
| - **Test single file**: `bun run test packages/core/__tests__/core.test.ts` | ||
| - **Typecheck**: `bun run typecheck` | ||
| - **CI (full)**: `bun run ci` | ||
|
|
||
| ## Code Style (Biome) | ||
| - **Indent**: 4 spaces | **Quotes**: double | **Semicolons**: always | **Trailing commas**: ES5 | ||
| - **Imports**: Use `type` keyword for type-only imports. Use `node:` prefix for Node.js built-ins (e.g., `node:child_process`) | ||
| - **File extensions**: Include `.ts` in local imports (e.g., `./setup.ts`) | ||
| - **Naming**: camelCase for functions/variables, PascalCase for types/interfaces | ||
| - **Tests**: Vitest with `describe`/`test`/`expect`. Files in `packages/*/__tests__/*.test.ts` | ||
| - **Error handling**: Use `try/catch` with `if (err instanceof Error)` checks | ||
| - **Source files**: Include copyright header: `/*! node-minify ... MIT Licensed */` | ||
|
|
||
| ### Linter Rules (from biome.json) | ||
| - `noForEach`: off (forEach allowed) | ||
| - `noParameterAssign`: off (parameter reassignment allowed) | ||
| - `noExplicitAny`: off (but prefer proper types when possible) | ||
| - Organize imports automatically on save | ||
|
|
||
| ## Architecture | ||
|
|
||
| This is a Bun monorepo for compressing JavaScript, CSS, and HTML files using various backends. | ||
|
|
||
| ### Package Structure | ||
|
|
||
| **Core packages** (in `/packages`): | ||
| - `core` - Main `minify()` function, orchestrates compression | ||
| - `utils` - Shared utilities (file operations, gzip sizing) | ||
| - `run` - Command execution wrapper for external tools | ||
| - `types` - TypeScript type definitions (not compiled) | ||
| - `cli` - Command-line interface | ||
|
|
||
| **Compressor packages** - Each wraps a specific minification library: | ||
| - JS: `esbuild`, `google-closure-compiler`, `oxc`, `swc`, `terser`, `uglify-js` | ||
| - CSS: `clean-css`, `cssnano`, `csso`, `esbuild`, `lightningcss` | ||
| - HTML: `html-minifier` | ||
| - Other: `jsonminify`, `no-compress` (passthrough) | ||
|
|
||
| **Deprecated** (still available but unmaintained upstream): | ||
| - `babel-minify` - Babel 6 only, use `terser` instead | ||
| - `uglify-es` - Unmaintained, use `terser` instead | ||
| - `yui` - Java-based, use modern alternatives instead | ||
| - `crass` - Unmaintained, use `lightningcss` or `clean-css` instead | ||
| - `sqwish` - Unmaintained, use `lightningcss` or `clean-css` instead | ||
|
|
||
| ### Dependencies | ||
|
|
||
| `core` depends on `utils` and `run`. All compressor packages depend on `core`. The build command (`bun run build`) builds `utils` and `run` first, then all other packages. | ||
|
|
||
| ### Package Pattern | ||
|
|
||
| All packages follow the same structure: | ||
| ```text | ||
| packages/<name>/ | ||
| ├── src/index.ts # Main export | ||
| ├── __tests__/ # Vitest tests | ||
| ├── package.json | ||
| └── tsconfig.json | ||
| ``` | ||
|
|
||
| Build: `tsdown` (configured via `tsdown.config.ts`) | ||
|
|
||
| ## Changesets | ||
|
|
||
| This project uses [Changesets](https://github.com/changesets/changesets) for versioning. | ||
| - **Add changeset**: `bun run changeset` (interactive prompt) | ||
| - **Version packages**: `bun run changeset:version` | ||
| - Required for any user-facing changes (features, fixes, breaking changes) | ||
|
|
||
| ## Adding a New Compressor | ||
|
|
||
| 1. Create `packages/<name>/` with standard structure | ||
| 2. Export async function matching `Compressor` type from `@node-minify/types` | ||
| 3. Function receives `{ settings, content }`, returns `{ code, map? }` | ||
| 4. Add tests using shared helpers from `tests/fixtures.ts` | ||
|
|
||
| Example compressor signature: | ||
| ```ts | ||
| import type { CompressorResult, MinifierOptions } from "@node-minify/types"; | ||
|
|
||
| export async function myCompressor({ | ||
| settings, | ||
| content, | ||
| }: MinifierOptions): Promise<CompressorResult> { | ||
| // ... minification logic | ||
| return { code: minifiedCode, map: sourceMap }; | ||
| } | ||
| ``` | ||
|
|
||
| ## Testing | ||
|
|
||
| Tests use shared fixtures from `tests/fixtures.ts`: | ||
| - `runOneTest({ options, compressorLabel, compressor })` - Run a single test case | ||
| - `tests` object contains test suites: `commonjs`, `commoncss`, `commonhtml`, `uglifyjs`, etc. | ||
|
|
||
| Types are in `packages/types/src/types.d.ts` (not `index.ts`). | ||
|
|
||
| ## Key Types | ||
|
|
||
| From `@node-minify/types`: | ||
| - `Settings` - User-facing options for `minify()` function | ||
| - `MinifierOptions` - What compressors receive (`{ settings, content }`) | ||
| - `CompressorResult` - What compressors return (`{ code, map? }`) | ||
| - `Compressor` - Function type: `(args: MinifierOptions) => Promise<CompressorResult>` | ||
|
|
||
| ## Common Patterns | ||
|
|
||
| ### File Operations (from @node-minify/utils) | ||
| ```ts | ||
| import { readFile, writeFile, getFilesizeInBytes } from "@node-minify/utils"; | ||
|
|
||
| const content = await readFile("src/app.js"); | ||
| await writeFile({ file: "dist/app.min.js", content, index: 0 }); | ||
| const size = await getFilesizeInBytes("dist/app.min.js"); | ||
| ``` | ||
|
|
||
| ### Deprecation Warnings | ||
| ```ts | ||
| import { warnDeprecation } from "@node-minify/utils"; | ||
|
|
||
| warnDeprecation("@node-minify/old-package", "Use @node-minify/new-package instead"); | ||
| ``` | ||
coderabbitai[bot] marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ### Async / Parallel Patterns | ||
|
|
||
| The codebase uses async functions for file operations and parallel compression. | ||
|
|
||
| #### Async File Operations | ||
| ```ts | ||
| import { | ||
| getContentFromFilesAsync, | ||
| isValidFileAsync, | ||
| } from "@node-minify/utils"; | ||
|
|
||
| // Async file reading (preferred for non-blocking IO) | ||
| const content = await getContentFromFilesAsync("src/app.js"); | ||
| const contents = await getContentFromFilesAsync(["src/a.js", "src/b.js"]); | ||
|
|
||
| // Async file validation | ||
| if (await isValidFileAsync("src/app.js")) { | ||
| // file exists and is readable | ||
| } | ||
| ``` | ||
|
|
||
| #### Parallel Compression (Array Inputs) | ||
|
|
||
| The core `minify()` function automatically processes array inputs in parallel: | ||
| ```ts | ||
| // Core handles parallelization internally via Promise.all | ||
| await minify({ | ||
| compressor: terser, | ||
| input: ["src/a.js", "src/b.js", "src/c.js"], | ||
| output: ["dist/a.min.js", "dist/b.min.js", "dist/c.min.js"], | ||
| }); | ||
| ``` | ||
|
|
||
| #### Custom Parallel Processing | ||
|
|
||
| For custom parallel operations: | ||
| ```ts | ||
| // Basic parallel execution | ||
| const results = await Promise.all( | ||
| files.map(async (file, index) => { | ||
| const content = await getContentFromFilesAsync(file); | ||
| return run({ settings, content, index }); | ||
| }) | ||
| ); | ||
|
|
||
| // With error handling per item (use allSettled) | ||
| const results = await Promise.allSettled( | ||
| files.map(file => compressSingleFile({ ...settings, input: file })) | ||
| ); | ||
| // Map results back to inputs | ||
| results.forEach((result, i) => { | ||
| if (result.status === "fulfilled") { | ||
| console.log(`${files[i]}: success`); | ||
| } else { | ||
| console.error(`${files[i]}: ${result.reason}`); | ||
| } | ||
| }); | ||
| ``` | ||
|
|
||
| #### Concurrency Limits | ||
|
|
||
| For large arrays, limit concurrent operations to avoid resource exhaustion: | ||
| ```ts | ||
| // Using p-limit (install: bun add p-limit) | ||
| import pLimit from "p-limit"; | ||
|
|
||
| const limit = pLimit(5); // max 5 concurrent | ||
| const results = await Promise.all( | ||
| files.map(file => limit(() => compressSingleFile({ ...settings, input: file }))) | ||
| ); | ||
|
|
||
| // Or batch processing | ||
| async function processBatches<T>(items: T[], batchSize: number, fn: (item: T) => Promise<unknown>) { | ||
| for (let i = 0; i < items.length; i += batchSize) { | ||
| await Promise.all(items.slice(i, i + batchSize).map(fn)); | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| **Guidelines:** | ||
| - Use `Promise.all` when all operations must succeed (fail-fast) | ||
| - Use `Promise.allSettled` when you need results for all items regardless of individual failures | ||
| - Limit concurrency to 5-10 for file operations, 2-4 for CPU-intensive compressions | ||
| - Always maintain 1:1 mapping between inputs and outputs for traceability | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| - **Build fails**: Run `bun run build:deps` first to build `utils` and `run` | ||
| - **Type errors**: Ensure `bun run build` completed; types come from compiled `dist/` | ||
| - **Test isolation**: Tests use `tests/tmp/` for output files (gitignored) | ||
| - **Clean rebuild**: `bun run clean && bun run build` | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.