Skip to content

Commit

Permalink
Add DeferringType
Browse files Browse the repository at this point in the history
friedkeenan committed Dec 29, 2024
1 parent cf5faac commit cd0ab33
Showing 6 changed files with 346 additions and 39 deletions.
1 change: 1 addition & 0 deletions docs/source/reference.rst
Original file line number Diff line number Diff line change
@@ -26,6 +26,7 @@ Types

~types.array
~types.default
~types.deferring
~types.enum
~types.misc
~types.numeric
56 changes: 25 additions & 31 deletions docs/source/tutorials/protocol_matching/context.rst
Original file line number Diff line number Diff line change
@@ -85,7 +85,7 @@ It is somewhat important that our ``FelinePacket.Context`` is default constructi

----

But how do we pass this information to our ``String`` type? This is where :class:`.Type.Context`\s come in. A :class:`.Type.Context` is basically just a wrapper for :class:`.Packet.Context`\s, giving only the additional information of what :class:`.Packet` the :class:`.Type` operation concerns, if any. A :class:`.Type.Context` can be acquired by calling the :class:`.Type.Context` constructor, optionally passing the relevant :class:`.Packet` instance and :class:`.Packet.Context`. The passed packet can be accessed through the :attr:`~.Type.Context.packet` attribute and the passed context can be accessed through the :attr:`~.Type.Context.packet_ctx` attribute, though you will likely not need to access it through that attribute as the attributes of the :class:`.Packet.Context` can be acquired through the constructed :class:`.Type.Context`, like so:
But how do we pass this information to our ``String`` type? This is where :class:`.Type.Context`\s come in. A :class:`.Type.Context` is basically just a wrapper for :class:`.Packet.Context`\s, giving only the additional information of what :class:`.Packet` the :class:`.Type` operation concerns, if any. A :class:`.Type.Context` can be acquired by calling the :class:`.Type.Context` constructor, optionally passing the relevant :class:`.Packet` instance and :class:`.Packet.Context`. The passed packet can be accessed through the :attr:`~.Type.Context.packet` attribute and the passed context can be accessed through the :attr:`~.Type.Context.packet_ctx` attribute, though you will likely not need to access it through that attribute as the attributes of the :class:`.Packet.Context` can be acquired directly through the constructed :class:`.Type.Context`, like so:

.. testcode::

@@ -186,9 +186,7 @@ So there we go, now we have a ``String`` type which packs and unpacks differentl

----

Is there a better way we could do this though? Yes, potentially. What we did is make our ``String`` code explicitly check the version stored in the ``ctx`` parameter, but we could abstract the logic out a bit further into a new :class:`.Type` which would represent the length of our string's encoded data. Then we could just have our ``String`` type call into that, being itself wholly unaware of any protocol versioning. So let's make that :class:`.Type`:

.. testcode::
Is there a better way we could do this though? Yes, potentially. What we did is make our ``String`` code explicitly check the version stored in the ``ctx`` parameter, but we could abstract the logic out a bit further into a new :class:`.Type` which would represent the length of our string's encoded data. Then we could just have our ``String`` :class:`.Type` call into that, being itself wholly unaware of any protocol versioning. So let's make that :class:`.Type`::

class StringDataLength(pak.Type):
@classmethod
@@ -205,7 +203,19 @@ Is there a better way we could do this though? Yes, potentially. What we did is

return pak.UInt16.pack(value, ctx=ctx)

Pretty simple, right? Let's put it to use now:
Here we just switch between using a :class:`.UInt8` or a :class:`.UInt16` to unpack and pack based on the version, but Pak actually provides a :class:`.DeferringType` facility for this sort of purpose already, which we can inherit from, and which will forward on the appropriate :class:`.Type` behavior to other operations beyond unpacking and packing as well. Here's how that would look:

.. testcode::

class StringDataLength(pak.DeferringType):
@classmethod
def _defer_to(cls, *, ctx):
if ctx.version < 1:
return pak.UInt8
return pak.UInt16

This then cuts down on a lot of overhead that we would otherwise have to write ourselves, and lets us focus on the actually meaningful logic of how to pick the appropriate :class:`.Type`. So let's put it to use now:

.. testcode::

@@ -263,7 +273,7 @@ We can actually at this point throw out all our custom ``String`` code and just

String = pak.PrefixedString(StringDataLength)

and come out with essentially the same functionality. This should give you a good idea about just how modular and composable :class:`.Type`\s can be.
and come out with essentially the same functionality. This all should give you a good idea about just how modular and composable :class:`.Type`\s can be.

Doing Better: Typelikes
***********************
@@ -280,35 +290,27 @@ This is where the concept of a "typelike" comes in. A typelike is an object that

But why are typelikes relevant here and how can we add our own?

Well, it would be really nice if instead of creating a whole ``StringDataLength`` type to handle the different protocol versions, we could instead have something like this::
Well, it would be really nice if instead of creating a whole ``StringDataLength`` :class:`.Type` to handle the different protocol versions, we could instead have something like this::

String = pak.PrefixedString({
0: pak.UInt8,
1: pak.UInt16,
})

This would be much more declarative, showing us very clearly that in version ``0``, strings are prefixed with a :class:`.UInt8` and in version ``1`` with a :class:`.UInt16`. This is where typelikes would help us, as we could make it so :class:`dict`\s are typelike, converting to a special type that forwards onto other types depending on the protocol version. Now, how would we go about this?
This would be much more declarative, showing us very clearly that in version ``0``, strings are prefixed with a :class:`.UInt8` and in version ``1`` with a :class:`.UInt16`. It would also be helpful if we wanted to reuse this API for other versioned :class:`.Type`\s our protocol could have. This is where typelikes would help us, as we could make it so :class:`dict`\s are typelike, converting to a special :class:`.Type` that forwards onto other :class:`.Type`\s depending on the protocol version. Now, how would we go about this?

First, we'll create the forwarding type; let's call it ``VersionedType``::
First, we'll create the forwarding :class:`.Type`; let's call it ``VersionedType``::

class VersionedType(pak.Type):
class VersionedType(pak.DeferringType):
# This will eventually be filled in with a specified dictionary.
version_types = None

@classmethod
def appropriate_type(cls, *, ctx):
def _defer_to(cls, *, ctx):
# Get the appropriate type for the version, converting any typelike results.
return pak.Type(cls.version_types[ctx.version])

@classmethod
def _unpack(cls, buf, *, ctx):
return cls.appropriate_type(ctx=ctx).unpack(buf, ctx=ctx)

@classmethod
def _pack(cls, value, *, ctx):
return cls.appropriate_type(ctx=ctx).pack(value, ctx=ctx)

And it's basically as simple as that. In a real protocol where you're gonna have more than two protocol versions, you would want a more refined way of getting the appropriate type than just indexing directly into the dictionary, but this is fine for our purposes. In the real world, you would also want to forward on more aspects of :class:`.Type`\s, but what we've done is sufficient for a tutorial.
And it's basically as simple as that. In a real protocol where you're gonna have more than two protocol versions, you would want a more refined way of getting the appropriate type than just indexing directly into the dictionary, but this is fine for our purposes.

Now how do we fill in that ``version_types`` attribute? Well, we can make it so calling ``VersionedType`` will fill it in, meaning we could use ``VersionedType`` like so::

@@ -323,23 +325,15 @@ Well, when :class:`.Type`\s get called, the :meth:`.Type._call` classmethod gets

.. testcode::

class VersionedType(pak.Type):
class VersionedType(pak.DeferringType):
# This will eventually be filled in with a specified dictionary.
version_types = None

@classmethod
def appropriate_type(cls, *, ctx):
def _defer_to(cls, *, ctx):
# Get the appropriate type for the version, converting any typelike results.
return pak.Type(cls.version_types[ctx.version])
@classmethod
def _unpack(cls, buf, *, ctx):
return cls.appropriate_type(ctx=ctx).unpack(buf, ctx=ctx)
@classmethod
def _pack(cls, value, *, ctx):
return cls.appropriate_type(ctx=ctx).pack(value, ctx=ctx)
@classmethod
def _call(cls, version_types):
# Make a subclass with the same name and with the 'version_types' attribute set.
@@ -372,7 +366,7 @@ We should now be able to use it like so:
assert StringDataLength.pack(2, ctx=ctx_version_0) == raw_data_version_0
assert StringDataLength.pack(2, ctx=ctx_version_1) == raw_data_version_1

And that's great, but we still haven't gotten to the API we set out for. To get there, we'll need to make :class:`dict`\s into typelikes that convert into a ``VersionedType``. To do this, we can use :meth:`.Type.register_typelike`, passing it the class of objects we want to be typelike (:class:`dict`), and a callable that will convert a :class:`dict` into a ``VersionedType``. Luckily, we just turned ``VersionedType`` into just that::
And that's great, but we still haven't gotten to the API we set out for. To get there, we'll need to make :class:`dict`\s into typelikes that convert into a ``VersionedType``. To do this, we can use :meth:`.Type.register_typelike`, passing it the class of objects that we want to be typelike (:class:`dict`), and a callable that will convert a :class:`dict` into a ``VersionedType``. Luckily, we just turned ``VersionedType`` into just that::

pak.Type.register_typelike(dict, VersionedType)

17 changes: 9 additions & 8 deletions pak/__init__.py
Original file line number Diff line number Diff line change
@@ -9,14 +9,15 @@
from .bit_field import *
from .dyn_value import *

from .types.type import *
from .types.array import *
from .types.numeric import *
from .types.string import *
from .types.enum import *
from .types.default import *
from .types.optional import *
from .types.misc import *
from .types.type import *
from .types.array import *
from .types.numeric import *
from .types.string import *
from .types.enum import *
from .types.default import *
from .types.optional import *
from .types.deferring import *
from .types.misc import *

from .packets import *

149 changes: 149 additions & 0 deletions pak/types/deferring.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
r""":class:`.Type`\s which defer their behavior to other :class:`.Type`\s."""

from .type import Type

__all__ = [
"DeferringType",
]

class DeferringType(Type):
r"""A :class:`.Type` which defers its behavior to other :class:`.Type`\s.
A :class:`DeferringType` will defer all of its marshaling behavior
to a certain :class:`.Type` depending on what it decides to return
from its :meth:`_defer_to` method.
This deferring of behavior is useful for instance in
protocols with multiple versions, where you may want
to have a :class:`.Packet` field act like a different
:class:`.Type` between different protocol versions.
:class:`DeferringType` should be preferred to custom :class:`.Type`\s
of a similar nature because :class:`DeferringType` will forward on
all relevant behavior, resulting in a more correct and ergonomic experience.
Examples
--------
>>> import pak
>>> class VersionedPacket(pak.Packet):
... class Context(pak.Packet.Context):
... def __init__(self, *, version):
... self.version = version
...
... super().__init__()
...
... def __hash__(self):
... return hash(self.version)
...
... def __eq__(self, other):
... if not isinstance(other, VersionedPacket.Context):
... return NotImplemented
...
... return self.version == other.version
...
>>> class VersionedInteger(pak.DeferringType):
... @classmethod
... def _defer_to(cls, *, ctx):
... # 'Int8' in version 0, 'Int16' in every other version.
... if ctx.version == 0:
... return pak.Int8
...
... return pak.Int16
...
>>> class MyPacket(VersionedPacket):
... number: VersionedInteger
...
>>> p = MyPacket(number=2)
>>>
>>> # The 'number' field is an 'Int8' in version 0.
>>> p.pack(ctx=VersionedPacket.Context(version=0))
b'\x02'
>>>
>>> # The 'number' field is an 'Int16' in version 1.
>>> p.pack(ctx=VersionedPacket.Context(version=1))
b'\x02\x00'
"""

class UnableToDeferError(ValueError, Type.UnsuppressedError):
"""An error indicating that there was no appropriate :class:`.Type` to defer to."""

@classmethod
def _defer_to(cls, *, ctx):
"""Gets the :class:`.Type` which the :class:`DeferringType` should defer to.
This method should be overridden by subclasses.
Parameters
----------
ctx : :class:`.Type.Context`
The context for the :class:`.Type`.
Returns
-------
subclass of :class:`.Type`
The appropriate :class:`.Type` to defer to
based on the ``ctx`` parameter.
Raises
------
:exc:`UnableToDeferError`
If the :class:`DeferringType` is unable
to defer to an appropriate :class:`.Type`.
"""

raise cls.UnableToDeferError(f"'{cls.__qualname__}' has not implemented deferring")

# NOTE: We cannot defer in our descriptor special methods
# as there is no context available there for us to inspect.

@classmethod
def _size(cls, value, *, ctx):
return cls._defer_to(ctx=ctx).size(value, ctx=ctx)

@classmethod
def _alignment(cls, *, ctx):
return cls._defer_to(ctx=ctx).alignment(ctx=ctx)

@classmethod
def _default(cls, *, ctx):
return cls._defer_to(ctx=ctx).default(ctx=ctx)

@classmethod
def _unpack(cls, buf, *, ctx):
return cls._defer_to(ctx=ctx).unpack(buf, ctx=ctx)

@classmethod
async def _unpack_async(cls, reader, *, ctx):
return await cls._defer_to(ctx=ctx).unpack_async(reader, ctx=ctx)

@classmethod
def _pack(cls, value, *, ctx):
return cls._defer_to(ctx=ctx).pack(value, ctx=ctx)

@classmethod
def _array_static_size(cls, array_size, *, ctx):
return cls._defer_to(ctx=ctx)._array_static_size(array_size, ctx=ctx)

@classmethod
def _array_default(cls, array_size, *, ctx):
return cls._defer_to(ctx=ctx)._array_default(array_size, ctx=ctx)

@classmethod
def _array_unpack(cls, buf, array_size, *, ctx):
return cls._defer_to(ctx=ctx)._array_unpack(buf, array_size, ctx=ctx)

@classmethod
async def _array_unpack_async(cls, reader, array_size, *, ctx):
return await cls._defer_to(ctx=ctx)._array_unpack_async(reader, array_size, ctx=ctx)

@classmethod
def _array_num_elements(cls, value, *, ctx):
return cls._defer_to(ctx=ctx)._array_num_elements(value, ctx=ctx)

@classmethod
def _array_ensure_size(cls, value, array_size, *, ctx):
return cls._defer_to(ctx=ctx)._array_ensure_size(value, array_size, ctx=ctx)

@classmethod
def _array_pack(cls, value, array_size, *, ctx):
return cls._defer_to(ctx=ctx)._array_pack(value, array_size, ctx=ctx)
5 changes: 5 additions & 0 deletions tests/test_packets/test_subpacket.py
Original file line number Diff line number Diff line change
@@ -200,6 +200,11 @@ class KnownID(TestID):

# An error should be raised upon encountering an unknown ID,
# instead of being suppressed like other exceptions.

with pytest.raises(pak.SubPacket.NoAvailableSubclassError, match="Unknown ID.+: 2"):
# Raw data is a known ID '1', then an unknown ID '2', then another known ID '1'.
TestID[None].unpack(b"\x01" + b"\x02" + b"\x01")

with pytest.raises(pak.SubPacket.NoAvailableSubclassError, match="Unknown ID.+: 2"):
# Raw data is a known ID '1', then an unknown ID '2', then another known ID '1'.
await TestID[None].unpack_async(b"\x01" + b"\x02" + b"\x01")
Loading

0 comments on commit cd0ab33

Please sign in to comment.