-
Notifications
You must be signed in to change notification settings - Fork 3.3k
feat(plugin-import-export): add new exportLimit, importLimit and per collection limit control and a few bugfixes #15405
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
📦 esbuild Bundle Analysis for payloadThis analysis was generated by esbuild-bundle-analyzer. 🤖 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds configurable limits for import and export operations to prevent excessive server resource usage, and fixes several bugs related to timezone handling, CSV export formatting, and configuration application. The limits can be set globally or per-collection and support dynamic functions based on request context.
Changes:
- Added
exportLimitandimportLimitconfiguration options (global and per-collection) - Fixed timezone companion fields being incorrectly included in exports
- Fixed
formatanddisableSavesettings not being applied to collection export configurations - Expanded test coverage for additional field types (checkbox, select, radio, email, textarea, code, point)
Reviewed changes
Copilot reviewed 85 out of 87 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| test/plugin-import-export/shared.ts | Added new collection slugs for testing limits and media uploads |
| test/plugin-import-export/seed/index.ts | Added seed data for new field types and media collection |
| test/plugin-import-export/payload-types.ts | Updated TypeScript types to include new collections, fields, and limit configurations |
| test/plugin-import-export/e2e.spec.ts | Added E2E tests for format enforcement, preview pagination, and limit enforcement |
| test/plugin-import-export/config.ts | Added configuration for timezone support and collections with limits |
| test/plugin-import-export/collections/PostsWithLimits.ts | Created new collection for testing limit enforcement |
| test/plugin-import-export/collections/Pages.ts | Added new field types for comprehensive testing |
| test/plugin-import-export/collections/Media.ts | Created media collection for upload field testing |
| packages/plugin-import-export/src/utilities/unflattenObject.ts | Improved code clarity with better variable naming and documentation |
| packages/plugin-import-export/src/utilities/resolveLimit.ts | New utility to resolve limit values (numbers or functions) |
| packages/plugin-import-export/src/utilities/parseCSV.ts | Improved comment clarity for CSV parsing logic |
| packages/plugin-import-export/src/utilities/getvalueAtPath.ts | Removed internal utility in favor of shared Payload function |
| packages/plugin-import-export/src/utilities/getSchemaColumns.ts | Added timezone companion field filtering and improved column ordering logic |
| packages/plugin-import-export/src/utilities/getPluginCollections.ts | Fixed application of non-override collection settings to base collection |
| packages/plugin-import-export/src/utilities/getImportFieldFunctions.ts | Cleaned up comments for better code clarity |
| packages/plugin-import-export/src/utilities/getExportFieldFunctions.ts | Added special handling for date fields to prevent incorrect timezone field inclusion |
| packages/plugin-import-export/src/utilities/flattenObject.ts | Added timezone field filtering and optimized field selection logic |
| packages/plugin-import-export/src/utilities/fieldToRegex.ts | New utility to create regex patterns for field path matching |
| packages/plugin-import-export/src/utilities/fieldToRegex.spec.ts | Comprehensive tests for field path regex generation |
| packages/plugin-import-export/src/utilities/collectTimezoneCompanionFields.ts | New utility to identify auto-generated timezone companion fields |
| packages/plugin-import-export/src/utilities/collectTimezoneCompanionFields.spec.ts | Tests for timezone companion field collection |
| packages/plugin-import-export/src/utilities/buildDisabledFieldRegex.ts | Rewritten to properly handle field names with underscores and block slugs |
| packages/plugin-import-export/src/utilities/buildDisabledFieldRegex.spec.ts | Tests for disabled field regex generation |
| packages/plugin-import-export/src/types.ts | Added Limit and LimitFunction types for configurable limits |
| packages/plugin-import-export/src/translations/languages/*.ts | Added translations for limit-related messages in all supported languages |
| packages/plugin-import-export/src/index.ts | Added limit configuration storage in collection custom properties |
| packages/plugin-import-export/src/import/handlePreview.ts | Added max limit resolution and enforcement in preview |
| packages/plugin-import-export/src/import/getImportCollection.ts | Added limit resolution for synchronous and job-based imports |
| packages/plugin-import-export/src/import/getCreateImportCollectionTask.ts | Added maxLimit field to import task definition |
| packages/plugin-import-export/src/import/createImport.ts | Added limit enforcement before processing import documents |
| packages/plugin-import-export/src/import/batchProcessor.ts | Improved variable naming and documentation for batch import processing |
| packages/plugin-import-export/src/exports/types.ts | Exported Limit types for external use |
| packages/plugin-import-export/src/export/handlePreview.ts | Added limit enforcement in export preview with timezone field filtering |
| packages/plugin-import-export/src/export/handleDownload.ts | Added format validation and limit resolution for downloads |
| packages/plugin-import-export/src/export/getFields.ts | Changed format field to read-only when forced format is configured |
| packages/plugin-import-export/src/export/getExportCollection.ts | Added limit resolution for synchronous and job-based exports |
| packages/plugin-import-export/src/export/getCreateExportCollectionTask.ts | Added maxLimit field to export task definition |
| packages/plugin-import-export/src/export/createExport.ts | Added limit enforcement logic with timezone field filtering and BOM for CSV |
| packages/plugin-import-export/src/components/ImportPreview/index.tsx | Replaced internal utility with shared Payload function |
| packages/plugin-import-export/src/components/ExportPreview/index.tsx | Added UI display for limit warnings when user limit exceeds max |
| packages/plugin-import-export/src/components/ExportPreview/index.scss | Added styling for limit warning message |
| docs/plugins/import-export.mdx | Added documentation for limit configuration and usage |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This PR adds
exportLimit,importLimitand per collectionlimitcontrol and fixes a few reported bugs around preview behaviour and data exports to CSV and expands the test coverage to all fields except uploads as those will be expanded upon in another PR. Some of these bugs are reported directly, not on Github.New limits control
Adds new exportLimit, importLimit and per collection limit control so you can force a limit per user if necessary for how many documents can be exported or imported in a single operation. This allows you to prevent certain users from causing excessive server/resource usage.
The config can be added top level to affect all operations or you can override on a per collection and per operation level, defaults to 0 (unlimited).
Bugs fixed:
disableSaveandformathaving no effect on the collection's export configurationChores:
payload/sharedoneformatpoint,checkbox,select,radio,email,textareaandcode