Type Hinting: Other Useful Features

The previous articles in this series were about some of the bigger concepts that we need to know to make good use of Python's type hinting feature. In this, the final article of the series, we look at some of the smaller features that are really useful to know.

Any

One of the reasons why Python is so popular is that it's dynamically typed. The very fact that you don't need to specify the types and a variable could potentially refer to different types is what gives the language its flexibility and power.

In fact, some computations can be expressed in a simple way with dynamic typing, but make your head spin when you want to type hint them properly.

Consider the simple example below. How should we type hint the parameters and return value?

def add(a, b):
    return a + b

At first glance, it may seem like this is simple. a is an int, b is an int  and the function adds them up and returns an int. But that's not all, because we could pass in two strings to the function and it would still work. Or we could pass two lists, or any number of other data types that implement the __add__ dunder method.

Along those lines, you could even pass an int for a and float for b and it would still work, giving a float output.

In situations like this, you can use Any to type hint it. This tells the type checker that the variable could be anything at all. You are then relying on run time validation to check the data – just like regular python without type hints.

from typing import Any

def add(a:Any, b:Any) -> Any:
    return a + b

This is a great escape door that python provides when things are starting to get a bit too complex.

Optional

Unlike many languages, Python does not have a concept of null. Every variable has to reference some object. None is also an object in python – it is a special singleton object – so when you set a variable to None, it is still referencing an object.

What this means is that Python's type hints become "non-nullable". If you define a data type for a variable, it has to have only that data type. You cannot set None to that variable because the None object is a different data type.

a: str = None
# ❌ mypy will give an error since None is not a string type

Therefore we need to explicitly state when a variable is allowed to have None as a value, and we can do this using the Union syntax that we saw in the previous article.

b: str | None = None # ✅ OK

Another way that we can define this is by using the Optional type as shown below

from typing import Optional

c: Optional[str] = None # ✅ OK

Both are equivalent and it is purely personal preference which syntax you want to use.

This need to explicitly state when the variable can contain None makes the type hinting syntax very robust. A big source of bugs is forgetting to handle when a variable is None. mypy can catch these errors as the code below demonstrates

# this function may end up returning None
def get_github_data(username: str) -> Optional[dict[str, Any]]:
    resp = requests.get(f"https://api.github.com/users/{username}")
    if resp.status_code == 200:
        return resp.json()
    return None
    
# this function will not accept None as an input
# because the param is not declared Optional
def print_user_id(data: dict[str, Any]) -> None:
    print(data['id'])

data = get_github_data('playfulpython')
print_user_id(data) #❌ mypy error. data might be None

if data is not None:
    print_user_id(data) #✅Correct. Call only if not None

TypedDict

As we saw before, the dict[K, V] type takes two generic type variables: K to represent the type of the key and V to represent the type of the value.

However, many times the dictionary contains data where different fields contain different types. This is especially true when working with JSON APIs which are represented as dictionaries.

Here is the response of github's user API from the code snippet above:

{
  "login": "playfulpython",
  "id": 100799242,
  "avatar_url": "https://avatars.githubusercontent.com/u/100799242?v=4",
  "url": "https://api.github.com/users/playfulpython",
  "site_admin": false,
  "name": null,
  "public_repos": 0,
  ...
}

As we can see, some fields are strings, some are integers, some boolean, some are Optional. How do we create a type to represent this kind of data?

This is where TypedDict comes into the picture. TypedDict allows us to create a dictionary type by specifying the types of every key.

from typing import Optional, TypedDict

class UserApiResponse(TypedDict):
    username: str
    id: int
    avatar_url: str
    url: str
    site_admin: bool
    name: Optional[str]
    public_repos: int
    ...

Now mypy knows the type of the individual keys and can use that to catch bugs in the code

# Return type is now UserApiResponse
def get_github_data(username: str) -> Optional[UserApiResponse]:
    resp = requests.get(f"https://api.github.com/users/{username}")
    if resp.status_code == 200:
        return resp.json()
    return None

data = get_github_data('playfulpython')
if data:
    user_id = data['id'] # user_id is an int
    print("ID:" + user_id) # ❌ mypy error. Cant use + for str & int
    name = data['name'] # name is Optional[str]
    print("Name:" + name) # ❌ name might be None

Casting

Final topic of this series – casting. Although mypy does some impressive analysis of the code, we must understand that it only analyses the type annotations. mypy cannot understand the meaning of the logic, and so there are a few places where mypy can come to wrong conclusion. Here is an example

from typing import Optional

def get_rate(all_items: dict[str, int], item: str) -> Optional[int]:
    # this returns None if item is not in dict
    return all_items.get(item)

The return type of get_rate is Optional[int]. This is because if the item is not present in the all_items dict then it will return None.

Now here is some code that uses the above function

all_items = {
    "jackfruit": 10,
    "mango": 15,
    "banana": 20
}

cart: list[str] = []
item = None
while item != '':
    item = input("Add an item (enter to stop): ")
    if item and item in all_items:
        cart.append(item)
        
rates: list[int] = [get_rate(all_items, item) for item in cart]
total = sum(rates)
print(f"Total = {total}")

The code asks the user to enter some items, which are added into the cart list. Then it uses get_rate function to get the rate of each item in the list and finally sums it up and prints the total.

When this code is given to mypy, it will complain on the following line

# ❌ mypy error
rates: list[int] = [get_rate(all_items, item) for item in cart]

The reason is if any item is not present in the dictionary, then get_rate will return None and this is not valid in a list which is supposed to be list[int].

However, while taking the input, we have ensured that the cart only contains items in the dictionary.

    if item and item in all_items:
        cart.append(item)

Therefore it is not possible that the item will not be in the dictionary, and in this particular code flow it is not possible that get_rate will return None.

mypy cannot work through the runtime logic of the code to come to this conclusion. We need to manually override the type that mypy has calculated. This is called casting the type, and the cast(new_type, value) function allows us to do this

from typing import cast

rates: list[int] = [cast(int, get_rate(all_items, item)) for item in cart]

We are casting the return value of get_rate to int, so mypy will consider the type as int instead of Optional[int] and it will not report an error in the code.

Casting is useful for scenarios like these where runtime logic can change what are the actual valid types for that particular code compared to the declared type. But we have to be careful: If there was some scenario where mypy was correctly pointing an error, the casting will override that and we might miss a bug.

Summary

That brings us to the end of this series on type hints. Type hinting in Python is extremely powerful. Because it is optional, we can easily get started with type hints by adding it to a few important places in the code. Type inferencing will propogate those type hints to other places in the code.

Hopefully this series of articles gave a good starting point to use type hints. We have covered all the basic features that most developers would need to use for day to day work. I would highly recommend using type hints, at least in the functions which are called at many other places in the code.