[stdlib] `SIMD.from_bytes()` and `SIMD.as_bytes()` #3966

msaelices · 2025-01-24T17:44:23Z

Similar to the Python's int.from_bytes() and int.to_bytes() one.

Signed-off-by: Manuel Saelices <[email protected]>

owenhilyard · 2025-01-25T20:46:02Z

docs/changelog.md

@@ -100,6 +100,10 @@ what we publish.

 ### Standard library changes

+- New `SIMD.from_bytes()` and `SIMD.as_bytes()` functions to convert a list of bytes
+  to an scalars and vice versa, accepting the endianess as an argument. Similar


to an scalars -> to scalars or to a list of scalars

Good catch! Fixed: msaelices@399b460

owenhilyard · 2025-01-25T20:47:01Z

stdlib/src/builtin/simd.mojo

@@ -1956,6 +1959,60 @@ struct SIMD[type: DType, size: Int](

        return bitcast[_integral_type_of[type](), size](self).cast[int_dtype]()

+    @staticmethod
+    fn from_bytes[
+        big_endian: Bool = False


This should probably default to is_big_endian().

+1, this is should be sys.info.is_big_endian()

Good suggestion! Done: msaelices@5852cb9

owenhilyard · 2025-01-25T20:47:11Z

stdlib/src/builtin/simd.mojo

+        return value
+
+    fn as_bytes[
+        big_endian: Bool = False


This should probably default to is_big_endian().

Done: msaelices@5852cb9

owenhilyard · 2025-01-25T20:49:12Z

stdlib/src/builtin/simd.mojo

+    @staticmethod
+    fn from_bytes[
+        big_endian: Bool = False
+    ](bytes: InlineArray[Byte, Self.type_len]) -> Scalar[type]:


Do we want this to be an InlineArray? Those are annoying to construct from a List or other type, thoughts on a Span variant?

That can be a follow up, for now the stricter one is fine with me

+1 to Span for the API type here.

How can we approach this? The Python int.from_bytes() allows an arbitrary large input because its output is an infinite sized int, like so:

>>> int.from_bytes(b'12321323123212133211213',"big") 4712049637989858926791184397276765665808674789488996659

If we allow an arbitrary large input, should the output stop being an scalar but a n-sized SIMD, being n power or two?

I would only consider the first Self.type_len bytes, ignoring everything else, and erroring if not enough bytes are present.

Let me try this and open a new PR if I feel useful. I don't like the current ergonomics using InlineArray

owenhilyard · 2025-01-25T20:50:10Z

stdlib/src/builtin/simd.mojo

+            value = byte_swap(value)
+
+        var ptr = UnsafePointer.address_of(value)
+        var array = InlineArray[Byte, Self.type_len](fill=0)


It might be better to leave this uninitalized so that LLVM can clearly see it can remove the zeroing.

That seems like more of an optimization, I'm not going to block on it right now

JoeLoser · 2025-01-27T17:19:08Z

@lsh do you mind TAL at this since you reviewed the original one before it got reverted internally? Thanks!

lsh · 2025-01-28T22:43:03Z

stdlib/src/builtin/simd.mojo

@@ -268,6 +269,8 @@ struct SIMD[type: DType, size: Int](
    alias _Mask = SIMD[DType.bool, size]

    alias element_type = type
+    alias type_len = type.sizeof()


quibble: I think this alias makes the methods more confusing rather than more clear.

Done: msaelices@513af8a

Signed-off-by: Manuel Saelices <[email protected]>

lsh · 2025-01-30T22:46:02Z

!sync

modularbot · 2025-02-01T01:46:50Z

✅🟣 This contribution has been merged 🟣✅

Your pull request has been merged to the internal upstream Mojo sources. It will be reflected here in the Mojo repository on the main branch during the next Mojo nightly release, typically within the next 24-48 hours.

We use Copybara to merge external contributions, click here to learn more.

modularbot · 2025-02-02T07:48:18Z

Landed in eb56605! Thank you for your contribution 🎉

msaelices added 3 commits January 24, 2025 18:38

Implement SIMD.from_bytes() and SIMD.to_bytes()

4f0e9b8

Signed-off-by: Manuel Saelices <[email protected]>

Tests for SIMD.as_bytes() and SIMD.from_bytes()

cd16ec4

Signed-off-by: Manuel Saelices <[email protected]>

Improve the SIMD.from_bytes() and SIMD.as_bytes() docstrings

e9c40f0

Signed-off-by: Manuel Saelices <[email protected]>

owenhilyard reviewed Jan 25, 2025

View reviewed changes

lsh reviewed Jan 28, 2025

View reviewed changes

Merge branch 'nightly' into simd-int-from-bytes

decbf64

skongum02 deleted the branch modular:main January 29, 2025 18:59

skongum02 closed this Jan 29, 2025

skongum02 reopened this Jan 29, 2025

skongum02 changed the base branch from nightly to main January 29, 2025 20:31

msaelices added 2 commits January 30, 2025 00:53

Merge branch 'main' into simd-int-from-bytes

454d4f2

Merge branch 'main' into simd-int-from-bytes

c69acd8

msaelices force-pushed the simd-int-from-bytes branch from 6a6ad76 to 67ba5cd Compare January 30, 2025 22:35

Fix changelog typo

399b460

Signed-off-by: Manuel Saelices <[email protected]>

msaelices force-pushed the simd-int-from-bytes branch from 67ba5cd to 399b460 Compare January 30, 2025 22:36

msaelices added 2 commits January 30, 2025 23:38

Better default value for the big_endian parameter

5852cb9

Signed-off-by: Manuel Saelices <[email protected]>

Slight improvement of the readability by removing the type_len alias

513af8a

Signed-off-by: Manuel Saelices <[email protected]>

modular-automation bot assigned lsh Jan 30, 2025

modularbot added the imported-internally Signals that a given pull request has been imported internally. label Jan 30, 2025

modularbot added the merged-internally Indicates that this pull request has been merged internally label Feb 1, 2025

modularbot added the merged-externally Merged externally in public mojo repo label Feb 2, 2025

modularbot closed this in eb56605 Feb 2, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[stdlib] `SIMD.from_bytes()` and `SIMD.as_bytes()` #3966

[stdlib] `SIMD.from_bytes()` and `SIMD.as_bytes()` #3966

msaelices commented Jan 24, 2025

owenhilyard Jan 25, 2025

msaelices Jan 30, 2025 •

edited

Loading

owenhilyard Jan 25, 2025

lsh Jan 28, 2025

msaelices Jan 30, 2025

owenhilyard Jan 25, 2025

msaelices Jan 30, 2025

owenhilyard Jan 25, 2025

lsh Jan 28, 2025

JoeLoser Jan 29, 2025

msaelices Jan 30, 2025 •

edited

Loading

owenhilyard Jan 31, 2025

msaelices Feb 2, 2025 •

edited

Loading

owenhilyard Jan 25, 2025

lsh Jan 28, 2025

JoeLoser commented Jan 27, 2025

lsh Jan 28, 2025

msaelices Jan 30, 2025

lsh commented Jan 30, 2025

modularbot commented Feb 1, 2025

modularbot commented Feb 2, 2025

[stdlib] SIMD.from_bytes() and SIMD.as_bytes() #3966

[stdlib] SIMD.from_bytes() and SIMD.as_bytes() #3966

Conversation

msaelices commented Jan 24, 2025

Choose a reason for hiding this comment

msaelices Jan 30, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

msaelices Jan 30, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

msaelices Feb 2, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JoeLoser commented Jan 27, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lsh commented Jan 30, 2025

modularbot commented Feb 1, 2025

modularbot commented Feb 2, 2025

[stdlib] `SIMD.from_bytes()` and `SIMD.as_bytes()` #3966

[stdlib] `SIMD.from_bytes()` and `SIMD.as_bytes()` #3966

msaelices Jan 30, 2025 •

edited

Loading

msaelices Jan 30, 2025 •

edited

Loading

msaelices Feb 2, 2025 •

edited

Loading