Skip to content

Conversation

@jheer
Copy link
Member

@jheer jheer commented Dec 9, 2024

  • Breaking: Drop custom Arquero JSON format with embedded schema. It provides little if any additional value (as the schema does not include type information), and I don't have any evidence that people use it.
  • Add support to load data from input readable streams.
  • Add full streaming readers for CSV, fixed width, and NDJSON formats.
  • Add support for gzip and deflate decompression of input streams.
  • Add fromArrowStream, fromCSVStream, fromFixedStream, and fromJSONStream methods for async stream parsing.
  • Expand fromJSON and toJSON to support a JSON type property, for control over row-oriented, column-oriented, and newline-delimited (NDJSON) formats.
  • Update build, including package.json use of the browser property, for more targeted node/web separation.
  • Update test cases to reflect changes.

@jheer jheer merged commit 2824cbf into main Feb 5, 2025
2 checks passed
@jheer jheer deleted the jh/stream branch February 5, 2025 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants