The difference between Bytes and Array[Byte]

[zxch3n]

Why the conversion between them is not free?

[peter-jerry-ye]

The Array[Byte] is mutable while Bytes are not.

[peter-jerry-ye]

If you have a free conversion then you might write this erronous code:

fn main {
  let bytes : Array[Bytes] = [b'\x00', b'\x01', b'\x02']
  let content : Bytes = bytes.to_bytes()
  bytes[1] = b'\x02'
  // now content is [0, 2, 2]
}

[peter-jerry-ye]

Apart from that, Bytes is more like FixedArray[Byte] instead of Array[Byte] as Array[Byte] is actually an object that has

• a larger FixedArray[Byte] than it is used
• extra properties to track its current usage

So Array[Byte] is analogous to Buffer, while the latter has more specific operations.

[zxch3n]

Got it, thanks. I opened this issue because I think it should also be included in the docs (in the website and the code)

[zxch3n]

BTW, could there be an API like Array::take_bytes() -> Bytes that clears the original Array, transfers the original content to Bytes, and thus implements zero cost conversion?

[peter-jerry-ye]

Do you have a concrete use case? I think it's possible for Buffer to implement it as long as the reserved size is exactly what you want, but I can not guarantee. For FixedArray[Byte], it is possible for certain backends as long as we are using reference counting so we can check if it is zero.

[zxch3n]

I think it would be great if there were a preferred way to handle Bytes. If the conversion is free, a function can always return a Array[Byte] for maximum compatibility.

[peter-jerry-ye]

Why not use a BytesView then?

[zxch3n]

When serializing an object, we may not know the size of the serialized data in advance. So we need to initialize it as Array[Byte] first, right? Also, the caller of the function may want to extend the array with other data. So Array[Byte] may be the most flexible return type if we can convert it into Bytes/BytesView with zero cost.

[peter-jerry-ye]

If you are trying to serializing data that you don't know the size in advance, you will definitely resize the buffer and introduce copying, and the final result can hardly be the exact size, which will introduce another copy. So I don't see how the 'zero cost' can be benefitial

[zxch3n]

It's a good point. But we usually don't need the capacity and length to be the same, right? My thought is if there is an ideal way to deal with bytes, there can be a convention so that people can share code more easily in the future.

BTW, will Bytes and Array[Byte] be compiled to Uint8Array when targeting JS? If it's number[], they will still be stored as 32-bit numbers in JS runtime, right?

[peter-jerry-ye]

I would suggest sharing with BytesView.

As of the ABI, I don't think it's stable yet (i.e. we won't make a commitment yet). I think the primitives are compiled to ArrayBuffer's @hackwaly is that the case?

[hackwaly]

We compiles Bytes / FixedArray[Byte] to Uint8Array. But due to implementation complexity, the backing buffer of Array[Byte] MAY be not Uint8Array. Some explanation here: In JavaScript Uint8Array has different prototype with Array.

I think you'd better to build a Array like growable container based on FixedArray[Byte] and using it to working on bytes. As we won't have it in near future.

[zxch3n]

Ah. I see. Thanks

[hackwaly]

Sorry. There's core/buffer I forgot.