Add mdb locale #6
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We inherited this issue from PostgreSQL. PostgreSQL uses glibc to sort strings. In version glibc=2.28, collations broke down badly (in general, there are no guarantees when updating glibc). Changing collations breaks indexes. Similarly, a cluster with different collations also behaves unpredictably. What and when something has changed in glibc can be found on https://github.com/ardentperf/glibc-unicode-sorting Also there is special postgresql-wiki https://wiki.postgresql.org/wiki/Locale_data_changes And you tube video https://www.youtube.com/watch?v=0E6O-V8Jato In short, the issue can be seen through the use of bash: ( echo "1-1"; echo "11" ) | LC_COLLATE=en_US.UTF-8 sort gives the different results in ubunru 18.04 and 22.04. There is no way to solve the problem other than by not changing the symbol order. We freeze symbol order and use it instead of glibc. Here the solution https://github.com/postgredients/mdb-locales. In this PR I have added PostgreSQL patch that replaces all glibc locale-related calls with a calls to an external libary. It activates using new configure parameter --with-mdblocales, which is off by default. Using custom locales needs libmdblocales1 package and mdb-locales package with symbol table. Build needs libmdblocales-dev package with headers.
Implement mdb-locales patch
* MDB admin patch & tests This patch introcudes new pseudo-pre-defined role "mdb_admin". Introduces 2 new function: extern bool mdb_admin_allow_bypass_owner_checks(Oid userId, Oid ownerId); extern void check_mdb_admin_is_member_of_role(Oid member, Oid role); To check mdb admin belongship and role-to-role ownership transfer correctness. Our mdb_admin ACL model is the following: * Any roles user or/and roles can be granted with mdb_admin * mdb_admin memeber can tranfser ownershup of relations, namespaces and functions to other roles, if target role in neither: superuser, pg_read_server_files, pg_write_server_files nor pg_execute_server_program. This patch allows mdb admin to tranfers ownership on non-superuser objects * f
This commit introduces new mdb internal role mdb_superuser. Role is capaple of: GRANT/REVOKE any set of priviledges to/from any object in database. Has power of pg_database_owner in any database, including: DROP any object in database (except system catalog and stuff) Role is NOT capaple of: Create database, role, extension or alter other roles with such priviledges. Transfer ownership to /pass has_priv of roles: PG_READ_ALL_DATA PG_WRITE_ALL_DATA PG_EXECUTE_SERVER_PROGRAM PG_READ_SERVER_FILES PG_WRITE_SERVER_FILES PG_DATABASE_OWNER Fix configure.ac USE_MDBLOCALES option handling Apply autoreconf stuff Set missing ok parameter ito true while acquiring mdb_superuser oid In regress tests, nobody creates mdb_superuser role, so missing ok is fine Allow mdb_superuser to have power of pg_database_owner Allow mdb_superuser to alter objects and grant ACl to objects, owner by pg_database_owner. Also, when acl check, allow mdb_supersuer use pg_database_owner role power to pass check
leborchuk
force-pushed
the
AddMDBLocale
branch
2 times, most recently
from
b358d88 to
2510f35
Compare
We inherited this issue from PostgreSQL. PostgreSQL uses glibc to sort strings. In version glibc=2.28, collations broke down badly (in general, there are no guarantees when updating glibc). Changing collations breaks indexes. Similarly, a cluster with different collations also behaves unpredictably. What and when something has changed in glibc can be found on https://github.com/ardentperf/glibc-unicode-sorting Also there is special postgresql-wiki https://wiki.postgresql.org/wiki/Locale_data_changes And you tube video https://www.youtube.com/watch?v=0E6O-V8Jato In short, the issue can be seen through the use of bash: ( echo "1-1"; echo "11" ) | LC_COLLATE=en_US.UTF-8 sort gives the different results in ubunru 18.04 and 22.04. There is no way to solve the problem other than by not changing the symbol order. We freeze symbol order and use it instead of glibc. Here the solution https://github.com/postgredients/mdb-locales. In this PR I have added PostgreSQL patch that replaces all glibc locale-related calls with a calls to an external libary. It activates using new configure parameter --with-mdblocales, which is off by default. Using custom locales needs libmdblocales1 package and mdb-locales package with symbol table. Build needs libmdblocales-dev package with headers.
leborchuk
force-pushed
the
AddMDBLocale
branch
from
f6e5de3 to
789969a
Compare
NJrslv
approved these changes
Dec 11, 2025
reshke
force-pushed
the
main
branch
2 times, most recently
from
03c99a9 to
a3e8a2e
Compare
Add branch protection rules for the REL_2_STABLE release branch to ensure all modifications must go through pull requests. Configuration added: * Require at least 2 approving reviews before merging * Require conversation threads to be resolved before merging This protects the release branch from direct pushes and enforces code review workflow for all changes.
* Extend workflow triggers to include `REL_2_STABLE` branch
* Modified 4 workflow files:
apache-rat-audit.yml
build-cloudberry.yml
build-dbg-cloudberry.yml
build-deb-cloudberry.yml
* Aligns test/build workflows between main and REL_2_STABLE
Please note maintainers must sync all the necessary commits from main to
REL_2_STABLE to keep workflows running successfully. This commit just
add the REL_2_STABLE as the target branch.
* ORCA: Fix memory leak in CWindowOids by adding destructor CWindowOids class was leaking three CMDIdGPDB objects (m_MDIdRowNumber, m_MDIdRank, m_MDDenseRank) that were allocated in the constructor but never released. Fixes ORCA unit test failures: - gporca_test_CXformTest - gporca_test_CConstExprEvaluatorDefaultTest
Also, while on it, beautify code to conform PG-style coding. Per coverity report 544476
Perl with higher version(e.g. 5.38.0) will produce different error message, compared with v5.34.0 We should handle these different versions.
The default value of join_collapse_limit was 20. When this value is set and the query contains about 20 joins (see added test), Postgres query optimizer cannot build a plan during hours and consumes a lot of memory, because the planner checks a lot of possible ways to join the tables. When join_collapse_limit is 8, the query plan is built in reasonable time.
Check IsSorted before Sort to reduce O(n log n) to O(n-1) comparisons for pre-sorted IN lists, improving ORCA optimization time.
Replace "format" with "format_type", because the format option is not supported
…r extended statistics. (apache#1550) Done more clean cherry-pick of CVE fix postgres/postgres@afe38fb Original commit message follows: ===== * Fix incorrect permissions-checking code for extended statistics. Commit a4d75c8 improved the extended-stats logic to allow extended stats to be collected on expressions not just bare Vars. To apply such stats, we first verify that the user has permissions to read all columns used in the stats. (If not, the query will likely fail at runtime, but the planner ought not do so.) That had to get extended to check permissions of columns appearing within such expressions, but the code for that was completely wrong: it applied pull_varattnos to the wrong pointer, leading to "unrecognized node type" failures. Furthermore, although you couldn't get to this because of that bug, it failed to account for the attnum offset applied by pull_varattnos. This escaped recognition so far because the code in question is not reached when the user has whole-table SELECT privilege (which is the common case), and because only subexpressions not specially handled by statext_is_compatible_clause_internal() are at risk. I think a large part of the reason for this bug is under-documentation of what statext_is_compatible_clause() is doing and what its arguments are, so do some work on the comments to try to improve that. Per bug #17570 from Alexander Kozhemyakin. Patch by Richard Guo; comments and other cosmetic improvements by me. (Thanks also to Japin Li for diagnosis.) Back-patch to v14 where the bug came in. Discussion: https://postgr.es/m/[email protected] --------- Co-authored-by: Tom Lane <[email protected]>
This commit introduces a new GitHub Actions workflow for building and testing Apache Cloudberry on Rocky Linux 8, enabling automated builds, RPM packaging, and regresssion testing alongside the existing Rocky 9 and Ubuntu 22.04 pipelines. Triggers: - Push to main branch - Pull requests modifying this workflow file - Scheduled: Every Monday at 02:00 UTC - Manual workflow dispatch with optional test selection
The `capture_output` argument for `subprocess.run` is not available in Python 3.6, which is the default on Rocky Linux 8. This commit replaces it with `stdout=subprocess.PIPE` and `stderr= subprocess.PIPE` to ensure backward compatibility. This enables `ic-good-opt-*` tests to pass on Rocky 8 environments. See: apache#1538
This commit fixes two issues preventing the `ic-contrib` regression tests from passing on Rocky Linux 8 environments (FIPS enabled). 1. Fixed regex matching for "Some PX error": The previous `init_file` rule missed a space in the error message pattern (`ERROR: Cannot use...`), causing FIPS error masking to fail. Added the missing space to correctly match the output. 2. Masked ephemeral line numbers: Different compilation environments (Rocky 8 vs 9) produce different line number references in error messages (e.g., pgcrypto.c:213 vs 215). Added `matchsubs` rules to mask these line numbers with `(pgcrypto.c:XXX)` and updated `expected/fips_2.out` to match, ensuring consistent test results across platforms. Changes: * Modified src/test/regress/init_file * Updated contrib/pgcrypto/expected/fips_2.out See: apache#1539
Similar to PR apache#1549, we will keep the pxf_fdw in `apache/cloudberry-pxf` as the latest version, so remove pxf_fdw test from this CI workflow file.
…tats_ext_exprs. (apache#1551) This is postgres/postgres@c342538 commit, applied to Cloudberry. There was no issues in apply, only changes are to gporca expected output original commit message follows === The catalog view pg_stats_ext fails to consider privileges for expression statistics. The catalog view pg_stats_ext_exprs fails to consider privileges and row-level security policies. To fix, restrict the data in these views to table owners or roles that inherit privileges of the table owner. It may be possible to apply less restrictive privilege checks in some cases, but that is left as a future exercise. Furthermore, for pg_stats_ext_exprs, do not return data for tables with row-level security enabled, as is already done for pg_stats_ext. On the back-branches, a fix-CVE-2024-4317.sql script is provided that will install into the "share" directory. This file can be used to apply the fix to existing clusters. Bumps catversion on 'master' branch only. Reported-by: Lukas Fittl Reviewed-by: Noah Misch, Tomas Vondra, Tom Lane Security: CVE-2024-4317 Backpatch-through: 14
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #ISSUE_Number
What does this PR do?
Type of Change
Breaking Changes
Test Plan
make installcheckmake -C src/test installcheck-cbdb-parallelImpact
Performance:
User-facing changes:
Dependencies:
Checklist
Additional Context
CI Skip Instructions