Skip to content

convert '~=' requirements specifier - #184

Open
M0ses wants to merge 6 commits into
openSUSE:masterfrom
M0ses:convert_requires
Open

M0ses wants to merge 6 commits into
openSUSE:masterfrom
M0ses:convert_requires

Conversation

@M0ses

@M0ses M0ses commented Nov 17, 2023

Copy link
Copy Markdown
Contributor

As described in #183 '~=' is not a valid version specifier for rpm and
deb packages.

This patch converts the specifier to valid version specifiers for
rpm/deb.

Examples:

abc ~= 1.1.1

  • in rpm:
BuildRequires: %{python_module abc >= 1.1.1} # Only for suse
Requires: python-abc >= 1.1.1, python-abc < 1.2
  • in deb:
Depends: python-abc (>= 1.1.1), python-abc (<< 1.2)

FIXES: #183
An schould also fix
FIXES: #93

@s-t-e-v-e-n-k

Copy link
Copy Markdown

Debian also has BuildDepends like BuildRequires, so you should extend this.

@M0ses

M0ses commented Nov 17, 2023

Copy link
Copy Markdown
Contributor Author

Debian Build-Depends should be fixed now by

20a8be8#diff-e1680efd70b630a38d8132340a9f77468b13b34f3b4162ceb1a8749bc1267665R8

@s-t-e-v-e-n-k

Copy link
Copy Markdown

@M0ses Hi, do you have time to rebase this to fix the conflicts?

@M0ses
M0ses force-pushed the convert_requires branch from ee62f7b to 664f317 Compare August 8, 2024 06:34

@danigm danigm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks mostly good to me, I've left some comments to try to simplify the code, but the new functionality looks good.

The change of the requires list to a list of list with a [PACKAGE, OP, VERSION] is asking for a change to create a new class to define a requirement and move all the operations to that class, but that's something that can be done after merging, just OOP stuff.

Comment thread py2pack/utils.py
Comment on lines +132 to +136
if "entry_points" not in data or not data['entry_points']:
return []
if isinstance(data["entry_points"], str):
eps = EntryPoints(EntryPoints._from_text(data["entry_points"]))
elif isinstance(data["entry_points"], dict):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change break the following code, because entry_points is not defined and it's used below.

And I think this change is not needed, it's doing the same. If data['entry_points'] is None the if isinstance(data["entry_points"], str): and elif isinstance(data["entry_points"], dict): will be false so it'll go to the else clause and will return []

Comment thread py2pack/requires.py
req += [req[0], '<', ".".join(v)]
out_list.append(req)

return out_list

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm understanding this code correctly, the return now is a list of lists, so we should update the documentation of this method to match the new way of producing the output.

Comment thread py2pack/requires.py
Comment on lines +115 to +116
v = req[2].split('.')
v.pop()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The version in a ~= requirement could be something without dots (abc ~= 3)? In that case, this can remove the version completely and will crash in the following line. But I'm not sure if that case is something that makes sense.

Comment thread py2pack/__init__.py
data['build_requires'] += ['wheel']

data["build_requires"] = [['setuptools']]
if _search_requires(data['build_requires'], 'setuptools') and not _search_requires(data['build_requires'], 'wheel'):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The search_requires function is not needed, this can be done with a list comprehension in a more pythonic way:

# Just the list of requires packages, without version information
require_names = [r[0] for r in data['build_requires']]
if 'setuptools' in require_names and 'wheel' not in require_names:

Comment thread py2pack/__init__.py Outdated
# setup jinja2 environment with custom filters
env = jinja2.Environment(loader=jinja2.FileSystemLoader(template_dir))

def _rpm_format_requires(req):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: This function does complex require formatting, it could be great to have some comment with examples of what it does, input -> output so it will be easier to understand in the future.

Comment thread py2pack/__init__.py
part2 = ", " + " ".join(req[3:])
return part1 + part2

def _parenthesize_version(req):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: This function does complex require formatting, it could be great to have some comment with examples of what it does, input -> output so it will be easier to understand in the future.

M0ses added 6 commits October 17, 2026 22:19
As described in openSUSE#183 '~=' is not a valid version specifier for rpm and
deb packages.

This patch converts the specifier to valid version specifiers for
rpm/deb.

Examples:

`abc ~= 1.1.1`

* in rpm:

```
BuildRequires: %{python_module abc >= 1.1.1} # Only for suse
Requires: python-abc >= 1.1.1, python-abc < 1.2
```

* in deb:

```
Depends: python-abc (>= 1.1.1), python-abc (<< 1.2)
```

FIXES: openSUSE#183
An schould also fix
FIXES: openSUSE#93

@mcepl mcepl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. We do need to translate ~= into operators RPM and Debian understand, but this revision has several correctness issues that need fixing before merging. I have left inline comments with reproductions.

The current CI failures include an undefined entry_points variable and a dropped Poetry version constraint. There are also template and conversion problems that the existing tests do not catch. Please add regression coverage for ~= conversion, repeated formatting of the same requirement, and RPM/Debian output, then get CI passing. A requirement-class refactor can wait; the output and crash fixes need to be part of this change.

BuildRequires: {{ req|rpm_format_requires("python-") }}
Requires: {{ req|rpm_format_requires("python-") }}
{%- endfor %}
{%- endif %}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This endif no longer has a matching if. Parsing the PR's Fedora template with Jinja raises TemplateSyntaxError: Encountered unknown tag 'endif' at line 23, whereas the base template parses successfully. There is another unmatched Jinja endif below, along with an orphaned RPM %endif. Please restore the conditional structure and the build dependencies needed by the retained %pyproject_* macros. The remaining dependency loops also need to use the new requirement format consistently.

Comment thread py2pack/__init__.py
env = jinja2.Environment(loader=jinja2.FileSystemLoader(template_dir))

def _rpm_format_requires(req, pfx):
req[0] = pfx + req[0]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please avoid modifying req in this formatter, including req[3] below. Mageia calls it twice on the same requirement, first for BuildRequires and then for Requires. With abc~=1.1.1, the second call produces python-python-abc >= 1.1.1, python-python-abc < 1.2. Formatting should leave the requirement unchanged and give the same result on repeated calls.

Comment thread py2pack/__init__.py
req[4] = '<<'
if req[4] == '>':
req[4] = '>>'
ver.append(req[3].lower() + " (" + " ".join(req[4:6]) + ")")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The second dependency does not receive the python- prefix supplied outside this filter by the .dsc template. For abc~=1.1.1, the rendered result is python-abc (>= 1.1.1), abc (<< 1.2). That upper bound constrains a different package. Both clauses need to name python-abc; please cover the complete rendered Debian dependency in a regression test.

Comment thread py2pack/utils.py
return []
if isinstance(data["entry_points"], str):
eps = EntryPoints(EntryPoints._from_text(data["entry_points"]))
elif isinstance(data["entry_points"], dict):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous review's undefined-variable issue is still present: this change removes the assignment to entry_points, but the comprehension below still calls entry_points.items(). Dictionary-style entry points now raise NameError, which also accounts for several current CI failures and the lint failure. The original implementation already returns an empty list for None, so reverting this unrelated change would address it.

Comment thread py2pack/__init__.py
nms = list(map(lambda n: n[0], req))
ret = []
for n in s:
if n[0] not in nms:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comparing only names drops requirements with different version constraints. For example, reject_pkg([['poetry-core', '==', '1.6.1']], [['poetry-core', '>=', '1.5.0']]) returns []. This removes Poetry's exact test build dependency merely because a weaker build-system requirement exists, and explains the Poetry fixture failure in CI. Please preserve distinct constraints or combine them without weakening the requirements.

Comment thread py2pack/requires.py
req[1] = '>='
v = req[2].split('.')
v.pop()
v[-1] = str(int(v[-1]) + 1)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Splitting the raw version on dots does not handle all valid PEP 440 versions. abc~=1.4.post1 is converted to an upper bound of <1.5, but the compatible-release upper bound is <2. abc~=1!1.4 raises ValueError when converting 1!1 to an integer. Please derive the bound from parsed release components and preserve the epoch. The changed tests currently contain no ~= cases, so please add ordinary numeric, post-release, and epoch examples.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

requires containing '~=' result in broken spec file Support for wheels

4 participants