fix(xnode): increase the length of parser column and config column

[acerDebugman]

Closes 6755743701
Closes 6723638357

Description

Issue(s)

• Close/close/Fix/fix/Resolve/resolve: Issue Link

Checklist

Please check the items in the checklist if applicable.

• Is the user manual updated?
• Are the test cases passed and automated?
• Is there no significant decrease in test coverage?

[gemini-code-assist]

Summary of Changes

Hello @acerDebugman, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a limitation in the maximum length of Xnode task parser and job configuration fields by substantially increasing their capacity. The core changes involve migrating the parser field to use dynamic memory allocation, implementing robust length validation checks, and ensuring proper memory management throughout the lifecycle of these fields. These updates enhance the system's flexibility in handling larger configurations and parser definitions, supported by a new comprehensive test case.

Highlights

• Increased Length Limits: The maximum allowed length for Xnode task parser and job configuration fields has been significantly increased from 16KB to 48KB.
• Dynamic Memory Allocation for Parser: The parser field within the SXnodeTaskOptions structure has been refactored from a fixed-size character array to a dynamically allocated char* pointer, enabling flexible length handling.
• Enhanced Error Handling and Memory Management: New error codes and length validation checks have been introduced for parser and configuration fields. Additionally, memory cleanup routines have been updated to properly deallocate dynamically allocated parser strings.
• New Test Case: A new Python test case (test_xnode_column_length) has been added to thoroughly validate the increased column lengths for Xnode task parsers and job configurations, including tests for empty strings.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• include/libs/nodes/cmdnodes.h
  • Changed parser field in SXnodeTaskOptions from a fixed-size array to char* to support dynamic allocation.
• include/util/taoserror.h
  • Added new error code TSDB_CODE_MND_XNODE_TASK_PARSER_TOO_LONG.
• include/util/tdef.h
  • Increased TSDB_XNODE_TASK_PARSER_LEN and TSDB_XNODE_TASK_JOB_CONFIG_LEN from 16384 to 48*1024 bytes.
• source/dnode/mnode/impl/src/mndXnode.c
  • Adjusted length comparison for jobObj.configLen and jobObj.reasonLen by adding + 1 to the respective maximum length constants, likely to account for null terminators.
• source/libs/nodes/src/nodesUtilFuncs.c
  • Added taosMemFreeClear(pOptions->parser); to nodesDestroyNode for QUERY_NODE_XNODE_TASK_OPTIONS to free dynamically allocated parser memory.
• source/libs/parser/src/parAstCreater.c
  • Improved error handling and memory cleanup by adding nodesDestroyNode calls to _err labels in createXnodeTaskWithOptions and dropXnodeResourceWhere.
  • Modified setXnodeTaskOption to perform length validation for 'parser'/'transform' options, allocate memory dynamically for pOptions->parser, and handle TSDB_CODE_MND_XNODE_TASK_PARSER_TOO_LONG error.
• source/libs/parser/src/parTranslater.c
  • Added a length check for the config option in translateCreateXnodeJob to prevent exceeding TSDB_XNODE_TASK_JOB_CONFIG_LEN.
• source/util/src/terror.c
  • Defined the error message string for TSDB_CODE_MND_XNODE_TASK_PARSER_TOO_LONG.
• test/cases/42-Xnode/test_xnode.py
  • Imported the string module.
  • Added test_xnode_column_length to verify the increased length limits for Xnode task parser and job config, including tests for maximum and empty string lengths.

Activity

• The pull request was initiated by acerDebugman.
• It aims to resolve issues identified by tickets 6755743701 and 6723638357.
• The author has included a checklist, indicating consideration for user manual updates, automated test cases, and test coverage, which suggests a thorough development process.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request successfully increases the length of the parser and config columns for xnode tasks and jobs. The change from a fixed-size array to a dynamically allocated pointer for the parser field is well-handled, with proper memory management for allocation and deallocation. The PR also includes valuable fixes for several memory leaks in the parser.

I have two main points of feedback:

1. A minor logic issue in calculating the parser string length for empty strings.
2. A more significant issue in the new test case where it uses a random number as a task ID, which will likely cause the test to fail or not test the intended functionality correctly.

Overall, the changes are good, but the test needs to be fixed to ensure correctness.