diff --git a/docs/source/reference.rst b/docs/source/reference.rst index 9256c10..45f2f6e 100644 --- a/docs/source/reference.rst +++ b/docs/source/reference.rst @@ -26,6 +26,7 @@ Types ~types.array ~types.default + ~types.deferring ~types.enum ~types.misc ~types.numeric diff --git a/docs/source/tutorials/protocol_matching/context.rst b/docs/source/tutorials/protocol_matching/context.rst index 043ce3f..8d0bf3a 100644 --- a/docs/source/tutorials/protocol_matching/context.rst +++ b/docs/source/tutorials/protocol_matching/context.rst @@ -85,7 +85,7 @@ It is somewhat important that our ``FelinePacket.Context`` is default constructi ---- -But how do we pass this information to our ``String`` type? This is where :class:`.Type.Context`\s come in. A :class:`.Type.Context` is basically just a wrapper for :class:`.Packet.Context`\s, giving only the additional information of what :class:`.Packet` the :class:`.Type` operation concerns, if any. A :class:`.Type.Context` can be acquired by calling the :class:`.Type.Context` constructor, optionally passing the relevant :class:`.Packet` instance and :class:`.Packet.Context`. The passed packet can be accessed through the :attr:`~.Type.Context.packet` attribute and the passed context can be accessed through the :attr:`~.Type.Context.packet_ctx` attribute, though you will likely not need to access it through that attribute as the attributes of the :class:`.Packet.Context` can be acquired through the constructed :class:`.Type.Context`, like so: +But how do we pass this information to our ``String`` type? This is where :class:`.Type.Context`\s come in. A :class:`.Type.Context` is basically just a wrapper for :class:`.Packet.Context`\s, giving only the additional information of what :class:`.Packet` the :class:`.Type` operation concerns, if any. A :class:`.Type.Context` can be acquired by calling the :class:`.Type.Context` constructor, optionally passing the relevant :class:`.Packet` instance and :class:`.Packet.Context`. The passed packet can be accessed through the :attr:`~.Type.Context.packet` attribute and the passed context can be accessed through the :attr:`~.Type.Context.packet_ctx` attribute, though you will likely not need to access it through that attribute as the attributes of the :class:`.Packet.Context` can be acquired directly through the constructed :class:`.Type.Context`, like so: .. testcode:: @@ -186,9 +186,7 @@ So there we go, now we have a ``String`` type which packs and unpacks differentl ---- -Is there a better way we could do this though? Yes, potentially. What we did is make our ``String`` code explicitly check the version stored in the ``ctx`` parameter, but we could abstract the logic out a bit further into a new :class:`.Type` which would represent the length of our string's encoded data. Then we could just have our ``String`` type call into that, being itself wholly unaware of any protocol versioning. So let's make that :class:`.Type`: - -.. testcode:: +Is there a better way we could do this though? Yes, potentially. What we did is make our ``String`` code explicitly check the version stored in the ``ctx`` parameter, but we could abstract the logic out a bit further into a new :class:`.Type` which would represent the length of our string's encoded data. Then we could just have our ``String`` :class:`.Type` call into that, being itself wholly unaware of any protocol versioning. So let's make that :class:`.Type`:: class StringDataLength(pak.Type): @classmethod @@ -205,7 +203,19 @@ Is there a better way we could do this though? Yes, potentially. What we did is return pak.UInt16.pack(value, ctx=ctx) -Pretty simple, right? Let's put it to use now: +Here we just switch between using a :class:`.UInt8` or a :class:`.UInt16` to unpack and pack based on the version, but Pak actually provides a :class:`.DeferringType` facility for this sort of purpose already, which we can inherit from, and which will forward on the appropriate :class:`.Type` behavior to other operations beyond unpacking and packing as well. Here's how that would look: + +.. testcode:: + + class StringDataLength(pak.DeferringType): + @classmethod + def _defer_to(cls, *, ctx): + if ctx.version < 1: + return pak.UInt8 + + return pak.UInt16 + +This then cuts down on a lot of overhead that we would otherwise have to write ourselves, and lets us focus on the actually meaningful logic of how to pick the appropriate :class:`.Type`. So let's put it to use now: .. testcode:: @@ -263,7 +273,7 @@ We can actually at this point throw out all our custom ``String`` code and just String = pak.PrefixedString(StringDataLength) -and come out with essentially the same functionality. This should give you a good idea about just how modular and composable :class:`.Type`\s can be. +and come out with essentially the same functionality. This all should give you a good idea about just how modular and composable :class:`.Type`\s can be. Doing Better: Typelikes *********************** @@ -280,35 +290,27 @@ This is where the concept of a "typelike" comes in. A typelike is an object that But why are typelikes relevant here and how can we add our own? -Well, it would be really nice if instead of creating a whole ``StringDataLength`` type to handle the different protocol versions, we could instead have something like this:: +Well, it would be really nice if instead of creating a whole ``StringDataLength`` :class:`.Type` to handle the different protocol versions, we could instead have something like this:: String = pak.PrefixedString({ 0: pak.UInt8, 1: pak.UInt16, }) -This would be much more declarative, showing us very clearly that in version ``0``, strings are prefixed with a :class:`.UInt8` and in version ``1`` with a :class:`.UInt16`. This is where typelikes would help us, as we could make it so :class:`dict`\s are typelike, converting to a special type that forwards onto other types depending on the protocol version. Now, how would we go about this? +This would be much more declarative, showing us very clearly that in version ``0``, strings are prefixed with a :class:`.UInt8` and in version ``1`` with a :class:`.UInt16`. It would also be helpful if we wanted to reuse this API for other versioned :class:`.Type`\s our protocol could have. This is where typelikes would help us, as we could make it so :class:`dict`\s are typelike, converting to a special :class:`.Type` that forwards onto other :class:`.Type`\s depending on the protocol version. Now, how would we go about this? -First, we'll create the forwarding type; let's call it ``VersionedType``:: +First, we'll create the forwarding :class:`.Type`; let's call it ``VersionedType``:: - class VersionedType(pak.Type): + class VersionedType(pak.DeferringType): # This will eventually be filled in with a specified dictionary. version_types = None @classmethod - def appropriate_type(cls, *, ctx): + def _defer_to(cls, *, ctx): # Get the appropriate type for the version, converting any typelike results. return pak.Type(cls.version_types[ctx.version]) - @classmethod - def _unpack(cls, buf, *, ctx): - return cls.appropriate_type(ctx=ctx).unpack(buf, ctx=ctx) - - @classmethod - def _pack(cls, value, *, ctx): - return cls.appropriate_type(ctx=ctx).pack(value, ctx=ctx) - -And it's basically as simple as that. In a real protocol where you're gonna have more than two protocol versions, you would want a more refined way of getting the appropriate type than just indexing directly into the dictionary, but this is fine for our purposes. In the real world, you would also want to forward on more aspects of :class:`.Type`\s, but what we've done is sufficient for a tutorial. +And it's basically as simple as that. In a real protocol where you're gonna have more than two protocol versions, you would want a more refined way of getting the appropriate type than just indexing directly into the dictionary, but this is fine for our purposes. Now how do we fill in that ``version_types`` attribute? Well, we can make it so calling ``VersionedType`` will fill it in, meaning we could use ``VersionedType`` like so:: @@ -323,23 +325,15 @@ Well, when :class:`.Type`\s get called, the :meth:`.Type._call` classmethod gets .. testcode:: - class VersionedType(pak.Type): + class VersionedType(pak.DeferringType): # This will eventually be filled in with a specified dictionary. version_types = None @classmethod - def appropriate_type(cls, *, ctx): + def _defer_to(cls, *, ctx): # Get the appropriate type for the version, converting any typelike results. return pak.Type(cls.version_types[ctx.version]) - @classmethod - def _unpack(cls, buf, *, ctx): - return cls.appropriate_type(ctx=ctx).unpack(buf, ctx=ctx) - - @classmethod - def _pack(cls, value, *, ctx): - return cls.appropriate_type(ctx=ctx).pack(value, ctx=ctx) - @classmethod def _call(cls, version_types): # Make a subclass with the same name and with the 'version_types' attribute set. @@ -372,7 +366,7 @@ We should now be able to use it like so: assert StringDataLength.pack(2, ctx=ctx_version_0) == raw_data_version_0 assert StringDataLength.pack(2, ctx=ctx_version_1) == raw_data_version_1 -And that's great, but we still haven't gotten to the API we set out for. To get there, we'll need to make :class:`dict`\s into typelikes that convert into a ``VersionedType``. To do this, we can use :meth:`.Type.register_typelike`, passing it the class of objects we want to be typelike (:class:`dict`), and a callable that will convert a :class:`dict` into a ``VersionedType``. Luckily, we just turned ``VersionedType`` into just that:: +And that's great, but we still haven't gotten to the API we set out for. To get there, we'll need to make :class:`dict`\s into typelikes that convert into a ``VersionedType``. To do this, we can use :meth:`.Type.register_typelike`, passing it the class of objects that we want to be typelike (:class:`dict`), and a callable that will convert a :class:`dict` into a ``VersionedType``. Luckily, we just turned ``VersionedType`` into just that:: pak.Type.register_typelike(dict, VersionedType) diff --git a/pak/__init__.py b/pak/__init__.py index 3e36b9b..1e7e4a6 100644 --- a/pak/__init__.py +++ b/pak/__init__.py @@ -9,14 +9,15 @@ from .bit_field import * from .dyn_value import * -from .types.type import * -from .types.array import * -from .types.numeric import * -from .types.string import * -from .types.enum import * -from .types.default import * -from .types.optional import * -from .types.misc import * +from .types.type import * +from .types.array import * +from .types.numeric import * +from .types.string import * +from .types.enum import * +from .types.default import * +from .types.optional import * +from .types.deferring import * +from .types.misc import * from .packets import * diff --git a/pak/types/deferring.py b/pak/types/deferring.py new file mode 100644 index 0000000..52e8172 --- /dev/null +++ b/pak/types/deferring.py @@ -0,0 +1,149 @@ +r""":class:`.Type`\s which defer their behavior to other :class:`.Type`\s.""" + +from .type import Type + +__all__ = [ + "DeferringType", +] + +class DeferringType(Type): + r"""A :class:`.Type` which defers its behavior to other :class:`.Type`\s. + + A :class:`DeferringType` will defer all of its marshaling behavior + to a certain :class:`.Type` depending on what it decides to return + from its :meth:`_defer_to` method. + + This deferring of behavior is useful for instance in + protocols with multiple versions, where you may want + to have a :class:`.Packet` field act like a different + :class:`.Type` between different protocol versions. + + :class:`DeferringType` should be preferred to custom :class:`.Type`\s + of a similar nature because :class:`DeferringType` will forward on + all relevant behavior, resulting in a more correct and ergonomic experience. + + Examples + -------- + >>> import pak + >>> class VersionedPacket(pak.Packet): + ... class Context(pak.Packet.Context): + ... def __init__(self, *, version): + ... self.version = version + ... + ... super().__init__() + ... + ... def __hash__(self): + ... return hash(self.version) + ... + ... def __eq__(self, other): + ... if not isinstance(other, VersionedPacket.Context): + ... return NotImplemented + ... + ... return self.version == other.version + ... + >>> class VersionedInteger(pak.DeferringType): + ... @classmethod + ... def _defer_to(cls, *, ctx): + ... # 'Int8' in version 0, 'Int16' in every other version. + ... if ctx.version == 0: + ... return pak.Int8 + ... + ... return pak.Int16 + ... + >>> class MyPacket(VersionedPacket): + ... number: VersionedInteger + ... + >>> p = MyPacket(number=2) + >>> + >>> # The 'number' field is an 'Int8' in version 0. + >>> p.pack(ctx=VersionedPacket.Context(version=0)) + b'\x02' + >>> + >>> # The 'number' field is an 'Int16' in version 1. + >>> p.pack(ctx=VersionedPacket.Context(version=1)) + b'\x02\x00' + """ + + class UnableToDeferError(ValueError, Type.UnsuppressedError): + """An error indicating that there was no appropriate :class:`.Type` to defer to.""" + + @classmethod + def _defer_to(cls, *, ctx): + """Gets the :class:`.Type` which the :class:`DeferringType` should defer to. + + This method should be overridden by subclasses. + + Parameters + ---------- + ctx : :class:`.Type.Context` + The context for the :class:`.Type`. + + Returns + ------- + subclass of :class:`.Type` + The appropriate :class:`.Type` to defer to + based on the ``ctx`` parameter. + + Raises + ------ + :exc:`UnableToDeferError` + If the :class:`DeferringType` is unable + to defer to an appropriate :class:`.Type`. + """ + + raise cls.UnableToDeferError(f"'{cls.__qualname__}' has not implemented deferring") + + # NOTE: We cannot defer in our descriptor special methods + # as there is no context available there for us to inspect. + + @classmethod + def _size(cls, value, *, ctx): + return cls._defer_to(ctx=ctx).size(value, ctx=ctx) + + @classmethod + def _alignment(cls, *, ctx): + return cls._defer_to(ctx=ctx).alignment(ctx=ctx) + + @classmethod + def _default(cls, *, ctx): + return cls._defer_to(ctx=ctx).default(ctx=ctx) + + @classmethod + def _unpack(cls, buf, *, ctx): + return cls._defer_to(ctx=ctx).unpack(buf, ctx=ctx) + + @classmethod + async def _unpack_async(cls, reader, *, ctx): + return await cls._defer_to(ctx=ctx).unpack_async(reader, ctx=ctx) + + @classmethod + def _pack(cls, value, *, ctx): + return cls._defer_to(ctx=ctx).pack(value, ctx=ctx) + + @classmethod + def _array_static_size(cls, array_size, *, ctx): + return cls._defer_to(ctx=ctx)._array_static_size(array_size, ctx=ctx) + + @classmethod + def _array_default(cls, array_size, *, ctx): + return cls._defer_to(ctx=ctx)._array_default(array_size, ctx=ctx) + + @classmethod + def _array_unpack(cls, buf, array_size, *, ctx): + return cls._defer_to(ctx=ctx)._array_unpack(buf, array_size, ctx=ctx) + + @classmethod + async def _array_unpack_async(cls, reader, array_size, *, ctx): + return await cls._defer_to(ctx=ctx)._array_unpack_async(reader, array_size, ctx=ctx) + + @classmethod + def _array_num_elements(cls, value, *, ctx): + return cls._defer_to(ctx=ctx)._array_num_elements(value, ctx=ctx) + + @classmethod + def _array_ensure_size(cls, value, array_size, *, ctx): + return cls._defer_to(ctx=ctx)._array_ensure_size(value, array_size, ctx=ctx) + + @classmethod + def _array_pack(cls, value, array_size, *, ctx): + return cls._defer_to(ctx=ctx)._array_pack(value, array_size, ctx=ctx) diff --git a/tests/test_packets/test_subpacket.py b/tests/test_packets/test_subpacket.py index 7b82dbd..fed9f49 100644 --- a/tests/test_packets/test_subpacket.py +++ b/tests/test_packets/test_subpacket.py @@ -200,6 +200,11 @@ class KnownID(TestID): # An error should be raised upon encountering an unknown ID, # instead of being suppressed like other exceptions. + with pytest.raises(pak.SubPacket.NoAvailableSubclassError, match="Unknown ID.+: 2"): # Raw data is a known ID '1', then an unknown ID '2', then another known ID '1'. TestID[None].unpack(b"\x01" + b"\x02" + b"\x01") + + with pytest.raises(pak.SubPacket.NoAvailableSubclassError, match="Unknown ID.+: 2"): + # Raw data is a known ID '1', then an unknown ID '2', then another known ID '1'. + await TestID[None].unpack_async(b"\x01" + b"\x02" + b"\x01") diff --git a/tests/test_types/test_deferring.py b/tests/test_types/test_deferring.py new file mode 100644 index 0000000..4227baf --- /dev/null +++ b/tests/test_types/test_deferring.py @@ -0,0 +1,157 @@ +import pak +import pytest + +class DeferringContext(pak.Packet.Context): + def __init__(self, *, use_raw_byte): + self.use_raw_byte = use_raw_byte + + super().__init__() + + def __hash__(self): + return hash(self.use_raw_byte) + + def __eq__(self, other): + if not isinstance(other, DeferringContext): + return NotImplemented + + return self.use_raw_byte == other.use_raw_byte + +class DeferringTest(pak.DeferringType): + @classmethod + def _defer_to(cls, *, ctx): + # We defer to either 'RawByte' or 'UInt16'. + + if ctx.use_raw_byte: + return pak.RawByte + + return pak.UInt16 + +ctx_raw_byte = pak.Type.Context(ctx=DeferringContext(use_raw_byte=True)) +ctx_uint16 = pak.Type.Context(ctx=DeferringContext(use_raw_byte=False)) + +async def test_deferring_type(): + await pak.test.type_behavior_both( + DeferringTest, + + (b"\xAA", b"\xAA"), + + static_size = 1, + alignment = 1, + default = b"\x00", + + ctx = ctx_raw_byte, + ) + + await pak.test.type_behavior_both( + DeferringTest, + + (1, b"\x01\x00"), + + static_size = 2, + alignment = 2, + default = 0, + + ctx = ctx_uint16, + ) + +async def test_deferring_type_array(): + await pak.test.type_behavior_both( + DeferringTest[2], + + (b"\xAA\xBB", b"\xAA\xBB"), + + static_size = 2, + alignment = 1, + default = b"\x00\x00", + + ctx = ctx_raw_byte, + ) + + await pak.test.type_behavior_both( + DeferringTest[2], + + ([1, 2], b"\x01\x00\x02\x00"), + + static_size = 4, + alignment = 2, + default = [0, 0], + + ctx = ctx_uint16, + ) + + await pak.test.type_behavior_both( + DeferringTest[pak.Int8], + + (b"\xAA\xBB", b"\x02\xAA\xBB"), + (b"", b"\x00"), + + static_size = None, + default = b"", + + ctx = ctx_raw_byte, + ) + + await pak.test.type_behavior_both( + DeferringTest[pak.Int8], + + ([1, 2], b"\x02\x01\x00\x02\x00"), + ([], b"\x00"), + + static_size = None, + default = [], + + ctx = ctx_uint16, + ) + + await pak.test.type_behavior_both( + DeferringTest[None], + + (b"\xAA\xBB\xCC", b"\xAA\xBB\xCC"), + + static_size = None, + default = b"", + + ctx = ctx_raw_byte, + ) + + await pak.test.type_behavior_both( + DeferringTest[None], + + ([1, 2, 3], b"\x01\x00\x02\x00\x03\x00"), + + static_size = None, + default = [], + + ctx = ctx_uint16, + ) + + class TestAttr(pak.Packet): + length: pak.Int8 + array: DeferringTest["length"] + + + assert TestAttr(length=2, ctx=ctx_raw_byte.packet_ctx).array == b"\x00\x00" + + await pak.test.packet_behavior_both( + (TestAttr(length=2, array=b"\xAA\xBB"), b"\x02\xAA\xBB"), + + ctx = ctx_raw_byte.packet_ctx, + ) + + assert TestAttr(length=2, ctx=ctx_uint16.packet_ctx).array == [0, 0] + + await pak.test.packet_behavior_both( + (TestAttr(length=2, array=[1, 2]), b"\x02\x01\x00\x02\x00"), + + ctx = ctx_uint16.packet_ctx, + ) + +async def test_deferring_type_cannot_defer_unbounded_array(): + # An error should be raised when there is no Type to defer + # to, instead of being suppressed like other exceptions. + + with pytest.raises(pak.DeferringType.UnableToDeferError, match="not implemented"): + pak.DeferringType[None].unpack(b"") + + with pytest.raises(pak.DeferringType.UnableToDeferError, match="not implemented"): + await pak.DeferringType[None].unpack_async(b"")