Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync fork #2

Closed
wants to merge 16 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,20 @@
# Changelog

## 3.10.14-post2

### Fixed

- Updated tests to correctly expect NaN/Infinity support and overflow handling
- Fix behavior on 0-dimensional arrays/tensors
- Fix behavior where Infinity/NaN were incorrectly being written as strings

## 3.10.14-post1

### Added

- Support serializing NaN and Infinity
- Support serializing PyTorch tensors when numpy serialization is enabled.


## 3.10.15

Expand Down
15 changes: 14 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ library for JSON and is more correct than the standard json library or other
third-party libraries. It serializes
[dataclass](https://github.com/ijl/orjson?tab=readme-ov-file#dataclass),
[datetime](https://github.com/ijl/orjson?tab=readme-ov-file#datetime),
[numpy](https://github.com/ijl/orjson?tab=readme-ov-file#numpy), and
[numpy](https://github.com/ijl/orjson?tab=readme-ov-file#numpy), [PyTorch](https://github.com/ijl/orjson?tab=readme-ov-file#pytorch), and
[UUID](https://github.com/ijl/orjson?tab=readme-ov-file#uuid) instances natively.

[orjson.dumps()](https://github.com/ijl/orjson?tab=readme-ov-file#serialize) is
Expand Down Expand Up @@ -798,6 +798,19 @@ orjson natively serializes `numpy.ndarray` and individual
`numpy.uintp`, `numpy.intp`, `numpy.datetime64`, and `numpy.bool`
instances.

### pytorch

orjson natively serializes PyTorch tensors (`torch.Tensor`) by converting them to numpy arrays. This requires both numpy and PyTorch to be installed, and the `OPT_SERIALIZE_NUMPY` option to be enabled:

```python
>>> import orjson, torch
>>> tensor = torch.tensor([[1, 2], [3, 4]])
>>> orjson.dumps(tensor, option=orjson.OPT_SERIALIZE_NUMPY)
b'[[1,2],[3,4]]'
```

The tensor must be on CPU and have a dtype that can be converted to a numpy array. GPU tensors are automatically moved to CPU before serialization.

orjson is compatible with both numpy v1 and v2.

orjson is faster than all compared libraries at serializing
Expand Down
30 changes: 12 additions & 18 deletions build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ fn main() {
println!("cargo:rustc-check-cfg=cfg(Py_3_8)");
println!("cargo:rustc-check-cfg=cfg(Py_3_9)");
println!("cargo:rustc-check-cfg=cfg(Py_GIL_DISABLED)");
println!("cargo:rustc-check-cfg=cfg(yyjson_allow_inf_and_nan)");

let python_config = pyo3_build_config::get();
for cfg in python_config.build_script_outputs() {
Expand Down Expand Up @@ -61,25 +62,18 @@ fn main() {
panic!("ORJSON_DISABLE_YYJSON and --features=yyjson both enabled.")
}
} else {
match cc::Build::new()
// Compile yyjson
cc::Build::new()
.file("include/yyjson/yyjson.c")
.include("include/yyjson")
.define("YYJSON_DISABLE_NON_STANDARD", "1")
.define("YYJSON_DISABLE_UTF8_VALIDATION", "1")
.define("YYJSON_DISABLE_UTILS", "1")
.define("YYJSON_DISABLE_WRITER", "1")
.try_compile("yyjson")
{
Ok(_) => {
println!("cargo:rustc-cfg=feature=\"yyjson\"");
}
Err(_) => {
if env::var("CARGO_FEATURE_YYJSON").is_ok() {
panic!(
"yyjson was enabled but the build failed. To build with a different backend do not specify the feature."
)
}
}
.compile("yyjson");

// Link against Python
let python_config = pyo3_build_config::get();
for cfg in python_config.build_script_outputs() {
println!("{cfg}");
}

println!("cargo:rustc-cfg=feature=\"yyjson\"");
println!("cargo:rustc-cfg=yyjson_allow_inf_and_nan");
}
}
1 change: 1 addition & 0 deletions data/roundtrip/roundtrip28.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[NaN]
1 change: 1 addition & 0 deletions data/roundtrip/roundtrip29.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[Infinity]
1 change: 1 addition & 0 deletions data/roundtrip/roundtrip30.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[NaN,Infinity,-Infinity]
Loading
Loading