Skip to content

Allow ignore_metadata when creating pandas #460

@delucchi-cmu

Description

@delucchi-cmu

Bug report

I've been debugging the following LSDB issue: astronomy-commons/lsdb#1265

Here's what I think is happening.

The data is older (imported with hats v0.4.0), and still contains pandas metadata inside the parquet files. While newer files will have this metadata suppressed, it's not a guarantee that these will be missing.

This automatically sets the index to _healpix_29, based on the pandas metadata. This is also true in the _ddf.meta.

I would like npd.read_parquet to return to me a nested frame that has no index set, and I believe that's not currently possible.

nested_pandas version: '0.6.8'

Before submitting
Please check the following:

  • I have described the situation in which the bug arose, including what code was executed, and any applicable data others will need to reproduce the problem.
  • I have included information about my environment, including the version of this package (e.g. nested_pandas.__version__)
  • I have included available evidence of the unexpected behavior (including error messages, screenshots, and/or plots) as well as a description of what I expected instead.
  • If I have a solution in mind, I have provided an explanation and/or pseudocode and/or task list.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions