Many tutorials on object-oriented programming conflate inheritance and subtyping. In fact, it's often considered part of the OO dogma that they should be conflated.

This is wrong for Python in a wide variety of ways.

Substitutability

Let's start with the Liskov Substitution Principle: A is a subtype of B iff instances of type A can substitute for instances of type B without affecting correctness or semantics. That's what it means to be a subtype.*

* Three things to note here, without getting into details (yet). First, mutability makes this more complicated than most people think at first—see the section on the ellipse-circle problem in the linked article. Second, languages like Python that have more complicated signatures—optional parameters, keyword-only parameters, vararg parameters, etc.—are harder to define rigorously, but all of those cases are easy to understand intuitively (e.g., a method that takes two integers can be overridden by a method that takes two integers and an optional string, because any call to the supertype's method will just passing two integers, which is valid for the subtype's method). Third, Python makes extensive use of what are implicitly union types and parametric types, but has no obvious way to represent them explicitly—a strstr-like function that returns None if not found is of type (str, str)->(int | NoneType); a read4k function could be described as of type (TextIOBase->str) | (IOBase->bytes) or of type (IOBase<T> -> String<T>). The MyPy-based static typing syntax for Python 3.5 or 3.6 will make some of this representable, but it's still not going to be completely parametric in the sense of ML or Haskell.

Nominal vs. structural substitutability

In many modern OO languages, only subclasses can be subtypes (and all subclasses should be subtypes, but this is usually only partly enforced). If I write a function that takes a File object, so I can read from it a line at a time (or 4K at a time), you can only call my function with something that inherits from File.*

In Python, this not only isn't necessary, it isn't idiomatic. If I write a function that wants to read a line at a time, I document that I want a file-like object that iterates a line at a time, and I don't verify that what I've gotten is-a file in any way, I just try to iterate it and expect to get strings. You can pass me an io.StringIO (which is basically a TextIOWrapper that isn't wrapping a file), or just a list of strings, or some type from a library that knew nothing about my library and vice-versa, and everything still works.

That's the difference between nominal and structural** (or behavioral***) subtyping, and that difference is half of what duck typing is all about. I'll get to the other half in a bit.

* A few languages (like C++) make things more complicated by allowing coercive typing: any type A that can be implicitly coerced to type B is a subtype of A, at least in some contexts… But let's ignore that.

** Structural subtyping can also include layout. In Python, since you generally store attributes in a dict rather than at specific offsets, layout is irrelevant beyond the names and types of the attributes. (Except, of course, that "generally" doesn't mean "always"; you can use __slots__, or inherit from ctypes.Structure, or write an extension module, or use nothing but descriptors in the class dictionary, or…) Also, in Python, classes generally don't even directly describe the attributes of their instances; they're created dynamically in __init__ (or even later), so checking that something is a subtype isn't even a sensible question except dynamically.

*** Behavioral subtyping has two different meanings: it can mean subtyping relationships that include semantics beyond those normally discussed in type theory, but it can also mean ad-hoc structural subtyping of exactly the kind Python uses.

What about isinstance?

But hold on. If I subtype without subclassing in Python, the result isn't really substitutable, because there's nothing stopping a function from calling isinstance and detecting that I'm not really what I'm claiming to be, right?

Well, that's true. But it's also true that even if I do subclass, there's nothing stopping a function from calling type (or looking at my __class__ attribute, or using the inspect module) and detecting that I'm not really what I'm claiming to be.

This is why idioms and conventions are important. In Java or C++, switching on isinstance-like checks (e.g., C++ dynamic_cast) is perfectly acceptable, because it gives you the exact same results that virtual function dispatch would give you. Switching on the exact type is strongly discouraged, because it effectively makes subtyping impossible.

In Python, isinstance is considered better than type, but still to be avoided whenever possible, precisely because it doesn't give you the same results that dynamic method lookup would give you, and it effectively makes subtyping impossible.

ABCs: practicality beats purity

So, structural subtyping good, nominal subtyping bad… But there really are some cases where it's useful to be able to check whether A is a subtype of B (this is just a specific case of "there are some cases where LBYL is more useful than EAFP"), and how else are you going to do that but isinstance?

But in most of those cases, you should use an abstract base class, not a normal base class. Don't check isinstance(arg, list), check isinstance(arg, collections.abc.MutableSequence). Note that this is exactly the same as using interfaces vs. base classes in Java or C#—the difference being that in Java, you always have to define an interface, while in Python you always can define an ABC, but usually don't have to.

But notice that ABC's don't require inheritance for subclassing. In fact, there are three ways to make a class a subclass of an ABC:
  • In the subclass's implementation, by inheriting from the ABC.
  • In the ABC's implementation, by writing a subclass hook that returns True for the subclass. (Note that this is usually done to implement structural subtyping, but it can just as easily be used to supertype a specific existing class that you can't modify.)
  • Outside of both implementations, in an application that just wants to make the two play together nicely, by registering the class with the ABC.
And meanwhile, Python doesn't even strictly require substitutability for ABCs. A TextIOBase is-a IOBase, even though IOBase.readline returns a bytes-like object and TextIOBase.readline returns a str-like object. Why? Well, as I'll explain later in the section on mixins, IOBase is more useful as a mixin class than as a subtyping class, and it's useful for IOBase to provide a default readline implementation for any subclass that provides readinto but doesn't provide readline. Speaking from a purist point of view, this means that TextIOBase isn't a subtype of IOBase at all. But from a practical point of view, it just means that readline isn't part of the IOBase type, even though it's part of the IOBase ABC.

What it all means

In (purist) theory, Python doesn't allow subtyping at all (because you can always call type), but in practice, Python uses subtyping all over the place—and as long as you write reasonably Pythonic code, it works wonderfully.

That's because duck typing, not subclassing, is the normal way to represent subtyping; inheritance is only one of three equally valid ways to implement subclassing; isinstance is used sparingly and only when you know you have ABCs to deal with; type is used only for debugging and very rare edge cases.

To anyone coming to Python from Lisp, Smalltalk, or JavaScript, this is all pretty obvious, but people coming from Java or C# (or, worse, languages that either mix nominal and duck typing haphazardly like Ruby, or mix them in a complicated way that many people never quite figure out like C++) end up writing non-Pythonic code because they think they need subclassing all over the place, when they actually don't.

Is this really subtyping at all?

The reason subtyping is important in OO in the first place is that it's how OO does polymorphism—that is, functions that can work with different types. You can write a function that parses an RFC822 header out of any file-like object, and it doesn't matter whether that object is a file on disk, a transport wrapper around a socket, or a wrapper around a string that you've built up in memory.

Above, I've described Python (and other duck-typed languages) as using implicit, dynamic, structural subtyping for its polymorphism. But there's a different way to look at it: You can just argue that Python's polymorphism isn't based on subtypes at all. Instead, its polymorphic functions are generic functions. The fact that many of those functions are written as OO-style methods and called using OO-style dot syntax doesn't necessarily mean there's any subtyping going on.

Well, I could point out that any language whose type system is powerful enough to represent both parametric types and union types has generic functions that can fully simulate OO methods (pause here for every Haskell programmer to say "duh"), although the converse is not true (pause again), and that duck typing automatically gives you a sufficiently powerful type system except for the compile-time part (kick off an async task to deal with the Haskell programmers who are now saying "the except is the whole point"*).

But subtyping is the most useful way to express much of what people frequently do with Python. For example, extend my RFC822 parser to an HTML handler: It reads an RFC822 header by reading line by line until it gets a blank line, then it does a binary read on the rest of the data. Sure, you _could_ describe that as taking type {__iter__: ()->Iterable<String>, detach: ()->{read: ()->Iterable<Byte>}}, but, at least for purposes of human documentation (as opposed to, say, compiler type checking or optimization), it's much more useful to say it takes any TextIOBase. Of course you could further document that it only uses part of the TextIOBase interface (which adds helpful information because that allows me to give you a duck-typed file-like object without having to implement the whole ABC), but you'd still want to start with the TextIOBase if you want humans to understand your code.

And anyway, if you don't want to think of Python as having subtyping at all, then you're obviously not going to implement subtyping with inheritance in Python, so I have nothing to convince you of.

* Briefly, they're forgetting the fact that both Haskell and Python—and many other dynamic languages, but no other remotely mainstream static language—allow you to write functions and data structures that are not expressible at all in ML, Java, etc. The fact that Haskell can catch errors at compile time, rather than runtime, and optimize at compile time rather than just by tracing-JIT (which works OK for time optimization, not so much for space…), etc. is definitely an extra feature on top of that, which Python doesn't have. But it doesn't add any expressiveness that Python doesn't have. (Now, a language that had a sufficiently powerful type system plus overloading might be able to express things that Python can't, but so far, C++ and Swift are the closest any mainstream languages have come to that, and they don't come very close.) Anyway, I can write further on this if anyone is interested and/or shaking their heads in anger and disbelief.

Not necessarily beautiful, but mutated

Getting back to the LSP, the novel idea that makes it so important is that it applies to mutable objects.

After all, for immutable objects, subtyping is simple: A is a subtype of B if, for every attribute in B, A has an attribute with the same name whose type is a subtype. For methods, being a subtype means all arguments* are supertypes, and the return value is a subtype. There are a few complexities for more complicated signatures like those Python allows, but, while they're hard to get down rigorously, they're intuitively obvious.**

But mutation makes everything trickier. First, for A to be a subtype of B, every directly mutable attribute must be of the same type, not a subtype, because you have to be able to assign the same values. And, while methods are obviously not usually mutable, they can be mutating, and a mutating method has to be able to mutate the object in the same way.***

The (in)famous example here is Circle and Ellipse. One of the standard examples for OO design is that a Circle is-a Ellipse whose width and height are always the same. Any non-mutating methods on an Ellipse will work perfectly well on a Circle. But if width is mutable, or there's a method that sets the width independently from the height, a Circle no longer is-a Ellipse. (If this isn't clear, consider an immutable stretched(aspect) method that returns a new Ellipse stretched to the given aspect ratio, vs. a mutable stretch(aspect) method that stretches self to the given aspect ratio. Obviously, Circle can implement the first one just fine, but can't implement the second.****)

For a more common real-life example, consider a sorted sequence—something that acts like a tuple, but always has all of its elements in sorted order. A sorted tuple can be do anything a tuple can do, and can be used anywhere a tuple can be used, but it can also do other things, like find all values within a certain range.***** But now consider a mutable sorted sequence—something that acts like a list, but always has all of its elements in sorted order. This is not a subtype of a mutable sequence (although it is still a subtype of sequence, and sorted sequence). The most basic mutable sequence operation is seq[idx] = value, with the postcondition that seq[idx] == value; a mutable sorted sequence can't provide that. Or consider calling seq.sort(key=different_key). In fact, almost any mutating sequence method makes no sense for a sorted sequence.

* Except for self, of course. Or the equivalent. Which can get pretty complicated when you're dealing with multiple dispatch—including the ad hoc multiple dispatch that Python's operator protocols provide—but let's not get into that here.

** For example, think about optional arguments: (int, int, int=0)->int is obviously a subtype of (int, int)->int.

*** To make that rigorous, you generally put it in terms of how subtyping handles either preconditions and postconditions, or invariants (or both), plus a history constraint.

**** Unless you incorporate type-changing into your language's OO model. In that case, if you define Circle such that it's no longer an invariant that every mutation leave it a Circle, only that every mutation leave it an Ellipse, then a Circle is-a Ellipse again. I believe people actually have written serious code like this in Lisp, and Python allows it as well (normal classes allow you to mutate the __class__ attribute on an instance), but I don't know of anyone taking advantage of it.

***** Well, a tuple can do that, it just takes linear time instead of logarithmic. And, if tuple slicing returned sub-views instead of copies, a sorted tuple could also return a view for this value-slicing operation, while a regular tuple would have to return a copy. That raises the whole question of to whether, how, and/or to what extent performance guarantees are part of a type, which I definitely don't want to get into here.

So what is inheritance for in Python?

Subtyping

Again, ABCs aren't completely useless for subtyping, and there are some cases where isinstance tests make sense.

And, to add to what I said before, they help document the requirements of a group of related functions, which would be handy even if they didn't help to check those requirements. Consider the C++ standard library, and its predecessor the STL: it relies on clearly-defined "concepts" that specify a number of abstract types, to allow pieces to be connected up without being coupled together, but the language has no support for putting the concepts into the actual implementation. (It came close to getting such support in C++11, and probably will have something eventually…) The fact that Python's type system is powerful enough to represent its concepts can't possibly hurt; it means you can inherit your class from collections.abc.Sequence, or register with it, instead of just adding a comment saying "# this class acts like a sequence". It's obvious, easy to spot, and even introspectable (which can be a pretty important thing for a language that's built around experimenting with components in a REPL). And within the ABCs themselves, inheritance is similarly useful. A Sequence is literally defined as a Sized, Iterable Container with some additional methods, not just in the documentation, but in the code.

Hooks

Often, the simplest way to provide some library functionality is to write the main code and provide a series of hooks for application developers to override whichever parts they want. And one obvious way to allow people to override code is method overriding.

For example, look at socketserver. A TCPServer object handles all the basic functionality of listening on a server socket, accepting client sockets, and processing their requests. There are a number of different levels at which the application can hook that processing, but generally you do it by subclassing BaseRequestHandler and overriding one or more of its methods. Or you can use an HTTPServer, which hooks TCPServer for you, and then subclass BaseHTTPRequestHandler and override some of its methods instead.

Of course the same thing could be done with a more functional hook interface—instead of subclassing BaseRequestHandler and overriding its handle method, you'd create a handle function that takes some kind of request parameter, then call set_handle_hook(my_handle_function). When the hook functions are all stateless or independent, this is identical, but when they have state that may be shared between them, the obvious place to put that state is in the self parameter—basically, the same reason you sometimes want to use objects instead of functions with an explicit cookie argument (or closures) in the first place.

Mixins (implementation sharing)

When you're learning OO theory, one of the first things you learn is that inheritance is for sharing interfaces, not implementation. But in many modern languages, that isn't true: you often want to build a class by composing behavior from multiple mixin or trait classes. Some languages have special syntax to do this, but in a language with proper multiple inheritance like Python, you can do this just by inheriting from mixins the same way you inherit from supertypes. For example, if you want to make a SocketServer concurrent, you can just inherit from ThreadingMixIn or ForkingMixIn as well as SocketServer (or write your own concurrency mixin that works differently).

And there's nothing stopping a class from being an interface and a mixin at the same time. In fact, this is common and idiomatic in Python. For example, most of the collection ABCs are written this way—on the one hand, they're a way to check types nominally, but on the other hand, they make it easier to implement structural subtypes. For example, you implement __getitem__, and the Sequence mixing automatically implements __iter__ for you. This isn't always the simplest way to go in Python (the total_ordering decorator shows another approach), but it's often a good design.

OO purists will argue that this is not proper OO—but again, practicality beats purity. Compare the various file-like objects in the stdlib, such as the one returned by socket.makefile, in 3.4 vs. 2.5. The modern version is almost always simpler and shorter, and it replaces the ad-hoc and inconsistent notion of "file-like object" with a set of well-specified (and nominally-checkable) interfaces.
Hybrid Programming
Hybrid Programming
5
Greenlets vs. explicit coroutines
Greenlets vs. explicit coroutines
6
ABCs: What are they good for?
ABCs: What are they good for?
1
A standard assembly format for Python bytecode
A standard assembly format for Python bytecode
6
Unified call syntax
Unified call syntax
8
Why heapq isn't a type
Why heapq isn't a type
1
Unpacked Bytecode
Unpacked Bytecode
3
Everything is dynamic
Everything is dynamic
1
Wordcode
Wordcode
1
For-each loops should define a new variable
For-each loops should define a new variable
4
Views instead of iterators
Views instead of iterators
2
How lookup _could_ work
How lookup _could_ work
2
How lookup works
How lookup works
7
How functions work
How functions work
2
Why you can't have exact decimal math
Why you can't have exact decimal math
2
Can you customize method resolution order?
Can you customize method resolution order?
1
Prototype inheritance is inheritance
Prototype inheritance is inheritance
1
Pattern matching again
Pattern matching again
The best collections library design?
The best collections library design?
1
Leaks into the Enclosing Scope
Leaks into the Enclosing Scope
2
Iterable Terminology
Iterable Terminology
8
Creating a new sequence type is easy
Creating a new sequence type is easy
2
Going faster with NumPy
Going faster with NumPy
2
Why isn't asyncio too slow?
Why isn't asyncio too slow?
Hacking Python without hacking Python
Hacking Python without hacking Python
1
How to detect a valid integer literal
How to detect a valid integer literal
2
Operator sectioning for Python
Operator sectioning for Python
1
If you don't like exceptions, you don't like Python
If you don't like exceptions, you don't like Python
2
Spam, spam, spam, gouda, spam, and tulips
Spam, spam, spam, gouda, spam, and tulips
And now for something completely stupid…
And now for something completely stupid…
How not to overuse lambda
How not to overuse lambda
1
Why following idioms matters
Why following idioms matters
1
Cloning generators
Cloning generators
5
What belongs in the stdlib?
What belongs in the stdlib?
3
Augmented Assignments (a += b)
Augmented Assignments (a += b)
11
Statements and Expressions
Statements and Expressions
3
An Abbreviated Table of binary64 Values
An Abbreviated Table of binary64 Values
1
IEEE Floats and Python
IEEE Floats and Python
Subtyping and Ducks
Subtyping and Ducks
1
Greenlets, threads, and processes
Greenlets, threads, and processes
6
Why don't you want getters and setters?
Why don't you want getters and setters?
8
The (Updated) Truth About Unicode in Python
The (Updated) Truth About Unicode in Python
1
How do I make a recursive function iterative?
How do I make a recursive function iterative?
1
Sockets and multiprocessing
Sockets and multiprocessing
Micro-optimization and Python
Micro-optimization and Python
3
Why does my 100MB file take 1GB of memory?
Why does my 100MB file take 1GB of memory?
1
How to edit a file in-place
How to edit a file in-place
ADTs for Python
ADTs for Python
5
A pattern-matching case statement for Python
A pattern-matching case statement for Python
2
How strongly typed is Python?
How strongly typed is Python?
How do comprehensions work?
How do comprehensions work?
1
Reverse dictionary lookup and more, on beyond z
Reverse dictionary lookup and more, on beyond z
2
How to handle exceptions
How to handle exceptions
2
Three ways to read files
Three ways to read files
2
Lazy Python lists
Lazy Python lists
2
Lazy cons lists
Lazy cons lists
1
Lazy tuple unpacking
Lazy tuple unpacking
3
Getting atomic writes right
Getting atomic writes right
Suites, scopes, and lifetimes
Suites, scopes, and lifetimes
1
Swift-style map and filter views
Swift-style map and filter views
1
Inline (bytecode) assembly
Inline (bytecode) assembly
Why Python (or any decent language) doesn't need blocks
Why Python (or any decent language) doesn't need blocks
18
SortedContainers
SortedContainers
1
Fixing lambda
Fixing lambda
2
Arguments and parameters, under the covers
Arguments and parameters, under the covers
pip, extension modules, and distro packages
pip, extension modules, and distro packages
Python doesn't have encapsulation?
Python doesn't have encapsulation?
3
Grouping into runs of adjacent values
Grouping into runs of adjacent values
dbm: not just for Unix
dbm: not just for Unix
How to use your self
How to use your self
1
Tkinter validation
Tkinter validation
7
What's the deal with ttk.Frame.__init__(self, parent)
What's the deal with ttk.Frame.__init__(self, parent)
1
Does Python pass by value, or by reference?
Does Python pass by value, or by reference?
9
"if not exists" definitions
"if not exists" definitions
repr + eval = bad idea
repr + eval = bad idea
1
Solving callbacks for Python GUIs
Solving callbacks for Python GUIs
Why your GUI app freezes
Why your GUI app freezes
21
Using python.org binary installations with Xcode 5
Using python.org binary installations with Xcode 5
defaultdict vs. setdefault
defaultdict vs. setdefault
1
Lazy restartable iteration
Lazy restartable iteration
2
Arguments and parameters
Arguments and parameters
3
How grouper works
How grouper works
1
Comprehensions vs. map
Comprehensions vs. map
2
Basic thread pools
Basic thread pools
Sorted collections in the stdlib
Sorted collections in the stdlib
4
Mac environment variables
Mac environment variables
Syntactic takewhile?
Syntactic takewhile?
4
Can you optimize list(genexp)
Can you optimize list(genexp)
MISRA-C and Python
MISRA-C and Python
1
How to split your program in two
How to split your program in two
How methods work
How methods work
3
readlines considered silly
readlines considered silly
6
Comprehensions for dummies
Comprehensions for dummies
Sockets are byte streams, not message streams
Sockets are byte streams, not message streams
9
Why you don't want to dynamically create variables
Why you don't want to dynamically create variables
7
Why eval/exec is bad
Why eval/exec is bad
Iterator Pipelines
Iterator Pipelines
2
Why are non-mutating algorithms simpler to write in Python?
Why are non-mutating algorithms simpler to write in Python?
2
Sticking with Apple's Python 2.7
Sticking with Apple's Python 2.7
Blog Archive
About Me
About Me
Loading
Dynamic Views theme. Powered by Blogger. Report Abuse.