DjangoCon Europe 2019: Building a custom model field from the ground up

Writeup of the DjangoCon Europe 2019 talk »Building a custom model field from the ground up« by Dmitry Dygalo

Dmitry Dygalo: Dmitry is a Tech Lead at kiwi.com in Prague. Started with Python in 2010 as a hobby during the university, he switched to a fulltime developer job after the graduation. He loves writing tests and cares about code maintainability. Hobbies: OpenSource and traveling.

Custom model fields: We want to map custom types to DB fields and interact with them! Le's look at django-money, which integrates money in the Django ORM, including forms, admin integration, template tags, DRF integration, currency rates.

Storage

Money consists of an amount as a decimal and a currency as a string. It includes arithmetic feeatures and localization for formatting currency output. We provide a MoneyField to do that. When creating a custom field, start with test cases (or use built-in test cases) for create/update/delete actions.

We have different storage options: separate fields (numeric and VARCHAR(3)) for amounts and strings are SQL compliant, but hard to manage. On the Python side, we'd use __get__, __set__ and __delete__ accessors to update attributes in __dict__.

An alternative are structured types, which is SQL compliant, and is a usual and good way of implementing custom fields in Django. This is not supported in MySQL, and has some overhead when accessing attributes. We'll need to implement from_db_value and get_prep_value to split up strings, and put them together again.

Summing up, this stage involves designing interfaces, discovering db queries and dbms support. Note: one field is simpler than multiple fields. Go for structured types if you can.

Queries

In terms of queries, we will need to support lookups, transformations, and expressions.

Reuse the code from existing lookups!

@MoneyField.register_lookup
class MoneyGreaterThan(models.lookups.GreaterThan)

    def as_sql(self, compiler, connection):
        return (
            # Insert SQL
        )

Add lookups for your field parts (money__currency='EUR'), for example. Basic expressions will work out of the box. For everything else, e.g. F expressions support, implement _add__, __sub__, __mul__, etc. on your model. And then implement custom expressions.

Summing up, this stage involves defining lookups and transformation behaviour unambiguously, mapping them to DB queries, and using the Lookup API for fun and profit (to build the desired queries). Extend magic methods on your entities for F support, and create custom expressions for your structured types, if you need them.

Extras

Migrations: By implementing deconstruct or @deconstructible, you'll add migration support.

Serialization: To support fixtures, you'll have to define a Serializer class and a Deserializer callable, plus support in get_prep_value. Register your serialization support via app_config with serializers.register_serializer in ready().

Validation: Subclass existing django.core.validator classes to add Max/min/whatever validators.

Admin: Structured types will pretty much just work out of the box – for other options, you'll have to implement some tricky hacks.

Summing up: Extend existing tools from Django. Use AppConfig to register your extensions, and think about which use cases are actually required for your field.

Summary

• Always start with the interface design and use it for tests.
• Explore underlying DB queries in different engines.
• Experiment with your implementation, choose the most simple and unambiguous.
• Try to evaluate possible consequences of your chosen approach.
• Django provides a lot of support for your modifications – make use of them!