Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADD crypto-fields #160

Open
wants to merge 19 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -425,6 +425,51 @@ class EmailSerializer(serializers.Serializer):
email = LowercaseEmailField()

```
## CryptoField Package
A set of generic common Djano Fields that automatically encrypt data for database amd encrypt for the purpose of usage in Django.
* *CryptoFieldMixin* - Mixin that implement encrypt/decrypt methods.
Example of how to use to create custom CryptoTextField
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## CryptoField Package
A set of generic common Djano Fields that automatically encrypt data for database amd encrypt for the purpose of usage in Django.
* *CryptoFieldMixin* - Mixin that implement encrypt/decrypt methods.
Example of how to use to create custom CryptoTextField
## CryptoField Package
A set of generic common Django Fields that automatically encrypts data for the database and for the purpose of usage in Django.
* *CryptoFieldMixin* - A mixin that implements encrypt/decrypt methods.
Example of how to use it to create the CustomCryptoTextField


```python
# models.py

from django.db import models

from drf_extra_fields.crypto_field import *

class CustomCryptoTextField(CryptoFieldMixin, models.TextField):
pass
```
- CryptoTextField - TextField inheriting Field
- CryptoCharField - CharField inheriting Field
- CryptoEmailField - EmailField inheriting Field
- CryptoIntegerField - IntegerField inheriting Field
- CryptoDateField - DateField inheriting Field
- CryptoDateTimeField - DateTimeField inheriting Field
- CryptoBigIntegerField - BigIntegerField inheriting Field
- CryptoPositiveIntegerField - PositiveIntegerField inheriting Field
- CryptoPositiveSmallIntegerField - PositiveSmallIntegerField inheriting Field
- CryptoSmallIntegerField - SmallIntegerField inheriting Field

**Settings.**
each CryptoField has 2 kwargs *salt_settings_env* and *password*.
* *salt_settings_env* - name of variable stored in settings.py file, which will be used as cryptographic salt. default: salt_settings_env = 'SECRET_KEY' if settings not set or no SECRET_KEY in setting default: salt = "Salt123!!!"
* *password* - password to be used in encryption process of given field (together with salt set globally) default = 'password'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**Settings.**
each CryptoField has 2 kwargs *salt_settings_env* and *password*.
* *salt_settings_env* - name of variable stored in settings.py file, which will be used as cryptographic salt. default: salt_settings_env = 'SECRET_KEY' if settings not set or no SECRET_KEY in setting default: salt = "Salt123!!!"
* *password* - password to be used in encryption process of given field (together with salt set globally) default = 'password'
### Settings
Each CryptoField has 2 kwargs: `salt_settings_env` and `password`.
* `salt_settings_env`: Name of the variable stored in the settings.py file, which will be used as cryptographic salt.
**Default**: `"SECRET_KEY"`
**Note**: If `salt_settings_env` does not exist in `settings.py` then `salt` will be `Salt123!!!`
* `password`: Password to be used in the encryption process of the given field (together with salt set globally).
**Default**: `"password"`


**Example.**

```python
# models.py

from django.db import models

from drf_extra_fields.crypto_field import *

class TestCryptoEmail(models.Model):

value = CryptoEmailField(salt_settings_env='NEW_SECRET_KEY', password='new_password')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**Example.**
```python
# models.py
from django.db import models
from drf_extra_fields.crypto_field import *
class TestCryptoEmail(models.Model):
value = CryptoEmailField(salt_settings_env='NEW_SECRET_KEY', password='new_password')
## Example
```python
# models.py
from django.db import models
from drf_extra_fields.crypto_field import CryptoEmailField
class User(models.Model):
encrypted_email = CryptoEmailField(salt_settings_env='NEW_SECRET_KEY', password='new_password')

```


CONTRIBUTION
=================
Expand Down
211 changes: 211 additions & 0 deletions drf_extra_fields/crypto_field.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
try:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please rename this file to crypto_fields.py? It makes more sense while importing:

from drf_extra_fields.crypto_fields import CryptoTextField

from functools import cached_property as property_decorator
except ImportError:
from builtins import property as property_decorator
import base64
import pickle
from django.utils.translation import gettext_lazy as _
from cryptography.fernet import Fernet
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
from django.conf import settings
from django.core.checks import Error
from django.core.exceptions import ImproperlyConfigured
from django.db import models
from django.utils.encoding import force_bytes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
try:
from functools import cached_property as property_decorator
except ImportError:
from builtins import property as property_decorator
import base64
import pickle
from django.utils.translation import gettext_lazy as _
from cryptography.fernet import Fernet
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
from django.conf import settings
from django.core.checks import Error
from django.core.exceptions import ImproperlyConfigured
from django.db import models
from django.utils.encoding import force_bytes
try:
from functools import cached_property as property_decorator
except ImportError:
from builtins import property as property_decorator
import base64
import pickle
from cryptography.fernet import Fernet
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
from django.conf import settings
from django.core.checks import Error
from django.core.exceptions import ImproperlyConfigured
from django.db import models
from django.utils.encoding import force_bytes
from django.utils.translation import gettext_lazy as _


__all__ = [
"CryptoFieldMixin",
"CryptoTextField",
"CryptoCharField",
"CryptoEmailField",
"CryptoIntegerField",
"CryptoDateField",
"CryptoDateTimeField",
"CryptoBigIntegerField",
"CryptoPositiveIntegerField",
"CryptoPositiveSmallIntegerField",
"CryptoSmallIntegerField",
]


def to_bytes(_obj):
if isinstance(_obj, bytes):
return _obj
elif isinstance(_obj, (bytearray, memoryview)):
return force_bytes(_obj)
else:
return pickle.dumps(_obj)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. This method doesn't work as expected for bytearray:
to_bytes(bytearray([1, 2, 3, 4, 5]))
# b"bytearray(b'\\x01\\x02\\x03\\x04\\x05')"

because force_bytes casts bytearray to str and then convert it to bytes. Here we don't need to use force_bytes, we can cast the object to bytes directly for bytearray and memoryview.

  1. What about calling this method _to_bytes(), since this is a specific method for crypto_field?

  2. What is the purpose of the underscore in _obj here? I guess using _to_bytes(obj) is fine.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the new version, I use validators rather than force casting types



class CryptoFieldMixin(models.Field):
"""
A Mixin that can be ued to convert standard django model field to encrypted binary. Fields are fully encrypted in
data base, but automatically readable in Django. Therefore there is no need for additional description of data,
it will be handheld automatically.

Cryptography protocol used in mixin: Fernet (symmetric encryption) provided by Cryptography (pyca/cryptography)
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"""
A Mixin that can be ued to convert standard django model field to encrypted binary. Fields are fully encrypted in
data base, but automatically readable in Django. Therefore there is no need for additional description of data,
it will be handheld automatically.
Cryptography protocol used in mixin: Fernet (symmetric encryption) provided by Cryptography (pyca/cryptography)
"""
"""
A Mixin that can be used to convert standard Django model fields to encrypted binary fields. Fields are fully encrypted in the
database, but will be automatically readable in Django. Therefore there is no need for additional description of data,
it will be handheld automatically.
Cryptography protocol used in mixin: Fernet (symmetric encryption) provided by Cryptography (pyca/cryptography).
"""


def __init__(self, salt_settings_env=None, password=None, *args, **kwargs):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

salt_settings_env sounds like an environment variable but it's not actually. Why not this is directly provided like password? Why there is an inconsistency?

I suggest to have DEFAULT_CRYPTOGRAPHY_SALT and DEFAULT_CRYPTOGRAPHY_PASSWORD variables in settings.py. These must be defined to be able to use crypto fields. If salt or password is not provided along with field definition these will be used. What do you think?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in an updated version, I use Django SECRET_KEY as salt but I kept the password as an optional parameter to make it more dynamic (the whole idea is to have something static - salt and dynamic - password as encryption ingredients)


if salt_settings_env and not isinstance(salt_settings_env, str):
raise ImproperlyConfigured("'salt_settings_env' must be a string")
self.salt_settings_env = salt_settings_env
self.password = "Password123!!!"

if password and not isinstance(password, (str, int)):
raise ImproperlyConfigured("'password' must be a string or int")

if password:
self.password = password

if kwargs.get("primary_key"):
raise ImproperlyConfigured(_(
"{} does not support primary_key=True.".format(self.__class__.__name__))
)
if kwargs.get("unique"):
raise ImproperlyConfigured(_(
"{} does not support unique=True.".format(self.__class__.__name__))
)
if kwargs.get("db_index"):
raise ImproperlyConfigured(_(
"{} does not support db_index=True.".format(self.__class__.__name__))
)
kwargs["null"] = True # should be nullable, in case data field is nullable.
kwargs["blank"] = True
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why null=False or blank=False cannot be set? 🤔 Also, here we are silently overriding kwargs provided by the user. If it is the case then we should raise an error if these are provided, as you did above.


self.salt = "Salt123!!!"

self.get_salt()

self._internal_type = "BinaryField"
super().__init__(*args, **kwargs)

def get_salt(self):
if self.salt_settings_env:
try:
self.salt = getattr(settings, self.salt_settings_env)
except AttributeError:
raise Error(_(
"salt_settings_env {} is not set in settings file".format(self.salt_settings_env))
)
else:
pass
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this method is redundant, we can keep this logic (if we still need to) in __init__.


def deconstruct(self):
name, path, args, kwargs = super().deconstruct()
# Only include kwarg if it's not the default (None)
if self.salt_settings_env:
kwargs["salt_settings_env"] = self.salt_settings_env
if self.password:
kwargs["password"] = self.password
return name, path, args, kwargs

def generate_password_key(self, password, salt):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method doesn't need to take password and salt as parameters, we can access them via self. Also, we can rename this method as _generate_password_key since it is a private method.

# password = b"password"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove this comment

kdf = PBKDF2HMAC(
algorithm=hashes.SHA256(),
length=32,
salt=to_bytes(salt),
iterations=100000,
)

key = base64.urlsafe_b64encode(kdf.derive(to_bytes(password)))
return key

@property_decorator
def fernet_key(self):
key = self.generate_password_key(self.password, self.salt)
return Fernet(key)

def encrypt(self, message):
b_message = to_bytes(message)
encrypted_message = self.fernet_key.encrypt(b_message)
return encrypted_message
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In get_db_prep_save and from_db_value the word value is being used. So what about renaming parameter message to value and variable b_message to something more clear and consistent, like value_in_bytes


def decrypt(self, encrypted_message):
b_message = to_bytes(encrypted_message)
decrypted_message = self.fernet_key.decrypt(b_message)
return decrypted_message
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comments with the encrypt method. ☝🏼


def get_internal_type(self):
return self._internal_type

def get_db_prep_save(self, value, connection, prepared=False):
if self.empty_strings_allowed and value == bytes():
value = ""
value = super().get_db_prep_value(value, connection, prepared=False)
if value is not None:
encrypted_value = self.encrypt(value)
return encrypted_value
# return connection.Database.Binary(encrypted_value)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about something like that?

Suggested change
def get_db_prep_save(self, value, connection, prepared=False):
if self.empty_strings_allowed and value == bytes():
value = ""
value = super().get_db_prep_value(value, connection, prepared=False)
if value is not None:
encrypted_value = self.encrypt(value)
return encrypted_value
# return connection.Database.Binary(encrypted_value)
def get_db_prep_save(self, value, connection, prepared=False):
if value:
value = self.encrypt(value)
return super().get_db_prep_value(value, connection, prepared)


def from_db_value(self, value, expression, connection):
if value is not None:
data = self.decrypt(value)
return pickle.loads(data)

@property_decorator
def validators(self):
# For IntegerField (and subclasses) we must pretend to be that
# field type to get proper validators.
self._internal_type = super().get_internal_type()
try:
return super().validators
finally:
self._internal_type = "BinaryField"


class CryptoTextField(CryptoFieldMixin, models.TextField):
pass


class CryptoCharField(CryptoFieldMixin, models.CharField):
pass


class CryptoEmailField(CryptoFieldMixin, models.EmailField):
pass


class CryptoIntegerField(CryptoFieldMixin, models.IntegerField):
pass


class CryptoPositiveIntegerField(CryptoFieldMixin, models.PositiveIntegerField):
pass


class CryptoPositiveSmallIntegerField(
CryptoFieldMixin, models.PositiveSmallIntegerField
):
pass


class CryptoSmallIntegerField(CryptoFieldMixin, models.SmallIntegerField):
pass


class CryptoBigIntegerField(CryptoFieldMixin, models.BigIntegerField):
pass


class CryptoDateField(CryptoFieldMixin, models.DateField):
pass


class CryptoDateTimeField(CryptoFieldMixin, models.DateTimeField):
pass


class CryptoBigIntegerField(CryptoFieldMixin, models.BigIntegerField):
pass


class CryptoDateField(CryptoFieldMixin, models.DateField):
pass


class CryptoDateTimeField(CryptoFieldMixin, models.DateTimeField):
pass
4 changes: 3 additions & 1 deletion drf_extra_fields/runtests/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,9 @@
# 'rest_framework.tests.users',
)

STATIC_URL = '/static/'
CRYPTO_SALT = "CryptoSalt123!!!"

STATIC_URL = "/static/"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's revert this.


PASSWORD_HASHERS = (
'django.contrib.auth.hashers.SHA1PasswordHasher',
Expand Down
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
Django >= 2.2
djangorestframework >= 3.9.2
cryptography
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please pin the version?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

version 3.2.1 is the last with python 3.5 support therefore I have pined that one

100 changes: 100 additions & 0 deletions tests/test_crypto.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
import datetime
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please rename this file to test_crypto_fields.py?


from django.test import TestCase

from drf_extra_fields.crypto_field import (
CryptoBigIntegerField,
CryptoCharField,
CryptoDateField,
CryptoDateTimeField,
CryptoEmailField,
CryptoIntegerField,
CryptoPositiveIntegerField,
CryptoPositiveSmallIntegerField,
CryptoSmallIntegerField,
CryptoTextField,
)


class PersonTest(TestCase):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
class PersonTest(TestCase):
class CryptoFieldsTests(TestCase):

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also what about creating a model consisting of all crypto fields and making some insert & update & delete operations to make sure that the whole integration is working?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tested in that matter in updated version

def test_get_prep_value(self):
text_field = "RandomTextField123!"
char_field = "RandomCharField123!"
email_field = "[email protected]"
int_field = -123
date_field = datetime.date(2001, 1, 1)
date_time_field = datetime.datetime(2001, 1, 1, 13, 00)
big_int_field = -9223372036854775808
positive_int_field = 123
positive_small_int_field = 1
small_int_field = -1

c_text = CryptoTextField()
c_char = CryptoCharField()
c_email = CryptoEmailField()
c_int = CryptoIntegerField()
c_date = CryptoDateField()
c_date_time = CryptoDateTimeField()
c_big_int = CryptoBigIntegerField()
c_positive_integer = CryptoPositiveIntegerField()
c_positive_small_integer = CryptoPositiveSmallIntegerField()
c_small_integer = CryptoSmallIntegerField()
self.assertEqual(
text_field,
c_text.get_prep_value(value=text_field),
)
self.assertEqual(
char_field,
c_char.get_prep_value(value=char_field),
)
self.assertEqual(
email_field,
c_email.get_prep_value(value=email_field),
)
self.assertEqual(
int_field,
c_int.get_prep_value(value=int_field),
)
self.assertEqual(
date_field,
c_date.get_prep_value(value=date_field),
)
self.assertEqual(
date_time_field,
c_date_time.get_prep_value(value=date_time_field),
)
self.assertEqual(
big_int_field,
c_big_int.get_prep_value(value=big_int_field),
)
self.assertEqual(
positive_int_field,
c_positive_integer.get_prep_value(value=positive_int_field),
)
self.assertEqual(
positive_small_int_field,
c_positive_small_integer.get_prep_value(value=positive_small_int_field),
)
self.assertEqual(
small_int_field,
c_small_integer.get_prep_value(value=small_int_field),
)

def test_get_db_prep_save(self):
c_text = CryptoTextField()
self.assertIs(
bytes,
type(c_text.get_db_prep_save(value="RandomTextField123!", connection=None)),
)

def test_to_python(self):
c_text = CryptoTextField()
self.assertEqual(str(""), c_text.to_python(""))
self.assertEqual(str("a"), c_text.to_python("a"))

def test_password_salt(self):
c_text = CryptoTextField()
c2_text = CryptoTextField(password="password_to_be_used_as_key")
self.assertEqual("Password123!!!", c_text.password)
self.assertEqual("Salt123!!!", c_text.salt)
self.assertEqual("password_to_be_used_as_key", c2_text.password)