Null-Safety in the Practical Type System (PTS)

First Published

2023-12-11

Author

Christian Neumanns

Editor

Tristano Ajmone

License

CC BY-ND 4.0


Note

This is part 5 in a series of articles titled How to Design a Practical Type System to Maximize Reliability, Maintainability, and Productivity in Software Development Projects.

It is recommended (but not required for experienced programmers) to read the articles in their order of publication, starting with Part 1: What? Why? How?.

Please be aware that PTS is a new, not-yet-implemented paradigm. As explained in section History of the article Essence and Foundation of the Practical Type System (PTS), PTS has been implemented in a proof-of-concept project, but a public PTS implementation isn't available yet — you can't try out the PTS source code examples shown in this article.

For a quick summary of previous articles you can read Summary of the Practical Type System (PTS) Article Series.


Original by biggerthanpluto via Pixabay

Introduction

The absence of a value is among the most important concepts a type system has to deal with — in one way or another.

Consider the scenario where the delivery date of a given order is still unknown. In this case, we can't assign a date to delivery_date, since there is no date available. We have to deal with the absence of a value.

Most software applications have to deal with many similar situations where no value is available for some object references.

A practical type system should therefore provide first class support to handle all cases gracefully. Handling the absence of a value should be easy, reliable, and maintainable.

This article explains how PTS aims to achieve this.

Approaches to Handle the "Absence of a Value"

Before showing how PTS handles the absence of a value, let's first look at common approaches.

Note

Readers only interested in the PTS approach can skip the next section.

Common Approaches

As far as I know, there are three common approaches to handle the absence of a value.

  • null

    Many programming languages use the symbol null to represent the absence of a value. Instead of null, some languages use nil, void, nothing, missing, etc., but the basic idea is the same.

    Note

    For a basic introduction to null you can read my article A Quick and Thorough Guide to null.

  • Null-Object Pattern

    In some languages there's no native support for handling the absence of a value.

    In such environments the null object pattern might be used. Wikipedia states:

    Instead of using a null reference to convey absence of an object (for instance, a non-existent customer), one uses an object which implements the expected interface, but whose method body is empty. A key purpose of using a null object is to avoid conditionals of different kinds, resulting in code that is more focused, quicker to read and follow - i e [sic] improved readability. One advantage of this approach over a working default implementation is that a null object is very predictable and has no side effects: it does nothing.

    Simple examples of applying the null object pattern would be to use zero for numbers, an empty string for object references of type string, and an empty collection for collection types.

    At first, this might seem to be a good solution, because not using null also means to get rid of the dreaded null pointer error.

    However, it turns out that the null object pattern creates more problems than it solves, and renders debugging more difficult — it's a poor man's solution. For more information and examples, you can read sections Using Zero Instead of Null and The Null Object Pattern in my article Why We Should Love 'null'.

  • Option/Maybe type

    This approach is based on the following premises:

    • null is not supported in the type system.

    • An Option type is used to handle the absence of a value. This type is also named Maybe, Optional, etc., but the basic idea is the same.

    One can think of Option/Maybe as a container that is either empty or contains a value.

    Most Option/Maybe implementations also provide specific functions/methods that are useful in the context of this type.

    In some languages the Option/Maybe type is also a monad. For example, Haskell provides the Maybe monad.

    Note

    For an introduction to monads (tailored to programmers who are unfamiliar with functional languages) you can read my article Simple Introduction to Monads — With Java Examples.

PTS Approach

PTS uses null. Yes, null!

This might come as a surprise, because modern programming languages tend to adopt the Option/Maybe approach.

Discussing the pros and cons of null vs Option/Maybe is beyond the scope of this article, but for an in-depth discussion you can read my article Null-Safety vs Maybe/Option — A Thorough Comparison. In a nutshell, that article shows:

  • Null-handling (if well implemented) is more convenient and practical for software developers than Option/Maybe, but it's more challenging to implement as a native feature in the language.

  • The Option/Maybe type is relatively easy to implement in the standard library.

If a choice must be made between easing the life of application developers or that of language engineers and compiler developers then, in the context of PTS, the preference goes to application developers. Therefore embracing null is a better approach in PTS.

Further reasons for choosing null have already been revealed in the previous article, titled Union Types in the Practical Type System.

However, null is a viable approach only if null-safety is guaranteed, as explained in the following section.

Why Do We Need Null-Safety?

History shows that the most frequent bug in many software applications is the infamous null pointer error, nowadays synonym with "the billion dollar mistake", a term coined by Sir Tony Hoare, Professor Emeritus, the inventor of null.

Professor John Sargeant from the Department of Computer Science, University of Manchester, puts it like this in his article Null pointer exceptions:

Of the things which can go wrong at runtime in Java programs, null pointer exceptions are by far the most common.

— Prof. John Sargeant

Here is an example of a null-pointer exception in Java:

        String name = null;
        int length = name.length();

Running this code will report the following run-time error, because method length() can't be executed on a null object:

Exception in thread "main" java.lang.NullPointerException: Cannot invoke "String.length()" because "name" is null
...

After reporting the error, the application aborts immediately. The consequences in production are unpredictable — they vary from harmless to disastrous.

Besides Java, other popular programming languages are also vulnerable to null pointer errors, for example C, C++, Go, JavaScript, Python, and Ruby. While modern C# has non-null types and specific null-checking features (since version 8), it still allows the use of null references, which can lead to null pointer errors.

Practice shows:

Many software applications written in non-null-safe languages are
copiously filled with null pointer bugs
lurking in the code and
waiting to hit at some unpredictable time in the future
causing undefined outcomes
ranging from harmless to very expensive disasters.

We need to eradicate the null pointer error.
We need null-safety.

Think about it: Null-safety eradicates the most common bug in languages that support null. Therefore null-safety is a most welcome feature — a crucial milestone in our pursuit of enhanced reliability, maintainability, and productivity.

Let's see how null-safety is achieved in PTS.

How Does It Work?

Fundamental Principles

PTS null-safety is based on the following principles:

  • PTS provides a built-in type dedicated to represent the absence of a value. This type is named null and it has a single value: null.

  • While null is a valid value for type null, it is invalid for all other types. In other words, all other types are non-null.

    For example, a function with an input parameter of type string cannot be called with null as input.

  • Union types are used to declare nullable object references (as covered already in the previous article titled Union Types in the Practical Type System). null can be assigned to a nullable object reference.

    For example, a function with an input parameter of type string or null can be called with null as input.

  • The compiler ensures that a method call on an object reference is only valid if the object reference is guaranteed to be non-null at run-time. Thus, null pointer errors cannot occur. The type system is null-safe.

    For example, foo.bar() is only valid if foo is guaranteed to be non-null at run-time.

  • A dedicated set of operators and statements facilitates null-handling and the development of null-safe code.

These are the fundamental design principles.

Now let's dig a bit deeper.

The null Type

Type null is a built-in type.

This type has a single value, also called null, in its set of allowed values. Thus the cardinality of type null is one.

null is used to represent the absence of a value.

For example, the assignment:

delivery_date = null

... states the absence of a date for delivery_date. No date is assigned to delivery_date.

Note

The code delivery_date = null doesn't tell us the reason for delivery_date pointing to null. The reason might be that a delivery date is not yet available. But the reason might also be that the delivery date has not yet been entered into the database. null just means there is no value available. It doesn't mean anything else, unless a specific meaning has been explicitly specified in a given context.

Type Hierarchy

The following class diagram shows the top types in the PTS type hierarchy:

Note

The convention in class diagrams is to style non-instantiable/abstract types (explained later) in italic.

At the top of the hierarchy sits type any — the root of all types in PTS. Types non_null and null both inherit from any. All other types inherit from non_null.

Note

Type inheritance has been briefly introduced in section Type Inheritance of a previous PTS article, titled Record Types in the Practical Type System. This topic isn't covered here — it's assumed that readers of this article are familiar with the concept of type inheritance.

Now let's have a closer look at these types.

Type any

The set of allowed values for the root type any is empty — its cardinality is zero. Simply put, type any doesn't have a value.

This implies that it's impossible to create an object of type any. Objects can only be created for some child-types/descendants of any. Type any is therefore called a non-instantiable (or abstract) type, and styled in italic in the above diagram.

Moreover, nobody is allowed to define child-types of any. Type any is a so-called sealed type. Its child-types are fixed: non_null and null.

The PTS source code for any looks like this:

type any \
    factories: none \            (1)
    child_types: non_null, null  (2)
.

(1) Instances of any can't be created.

(2) non_null and null are the only child-types.

Type non_null

Type non_null is a child of type any.

The set of allowed values for type non_null is empty too — its cardinality is zero. Simply put, type non_null doesn't have a value.

Like type any, type non_null is a non-instantiable type too. It's also a sealed type, as will be explained in the next PTS article.

Type null

Type null is a child of type any.

As seen already in a previous section, type null has a single value, called null, in its set of allowed values. The cardinality of type null is one. Simply put, type null has only one value: null.

Unlike types any and non_null, type null is an instantiable (or concrete) type.

Like any and non_null, null is also a sealed type. There are no child-types of null, and nobody is allowed to create child-types.

Other Types

All remaining built-in types not shown in the diagram (string, number, list, etc.), as well as all user-defined types (e.g. customer, supplier, product), inherit from non_null.

Rules

According to the important Liskov substitution principle, any type in a type inheritance tree is compatible with all its parent types.

Applying this principle to the above type inheritance tree implies the following rules:

  • Any value of any type can be assigned to an object reference declared to be of type any.

    For example, a function with an input parameter of type any can be called with the following values: null, "foo", 123, or any other value of any other type.

  • null is an invalid value for object references of type non_null (or any of its descendants).

    For example:

    • A function with an input parameter of type non_null cannot be called with null as input. But any value of every other type ("foo", 123, etc.) is allowed as input.

    • A function with an input parameter type that's a descendant of non_null (e.g. string, customer) cannot be called with null as input.

      Note

      This is exactly the opposite of what's allowed in many popular languages, such as C, Java, JavaScript, Python, Ruby, where every object reference can be null. In Java, for example, null can be assigned to an input parameter of type String.

  • It doesn't make sense to declare an object reference of type null. Therefore the compiler doesn't allow it.

    For example, a function input parameter cannot be of type null.

    Type null can only be used as a member of a union type, as explained in the next section.

Nullable Object References

A union type (see Union Types in the Practical Type System) is used to declare a nullable object reference. An object reference is nullable if its type is a union type that contains member null (e.g. string or null). In that case the value null is valid.

For example, if function foo has an input parameter of type string or null then calling it with null as input is valid, as shown in the following code:

function foo ( string or null )
    // body
.

foo ( "bar" ) // ok
foo ( null )  // ok

On the other hand, null is not allowed if the input parameter type is simply string:

function foo ( string )
    // body
.

foo ( "bar" ) // ok
foo ( null )  // compile-time error

Helpful Operators

In this section we'll have a look at useful operators that facilitate null-handling.

Operator is

Note

The is operator has already been introduced in section Operator is of the previous PTS article titled Union Types in the Practical Type System.

This section only covers its usage in the context of null-handling.

The is operator checks whether an expression is of a given type. The result is a boolean value. For example, "foo" is string evaluates to true, while 123 is string evaluates to false.

Hence, this operator is useful to check whether a given expression evaluates to null. Here is an example:

if customer.email_address is null then
    write_line ( "No email address!" )
else
    send_email ( email_address )
.

Instead of is, we can use is not to invert the check. Hence the above code can also be written like this:

if customer.email_address is not null then
    send_email ( email_address )
else
    write_line ( "No email address!" )
.

Besides using is in an if statement, we can also use it in an if expression:

const has_email = if customer.email_address is null then "no" else "yes"
write_line ( """Customer has email: {{has_email}}""" )

The Safe Navigation Operator (?)

The Wikipedia article Safe navigation operator states:

In object-oriented programming, the safe navigation operator (also known as optional chaining operator, safe call operator, null-conditional operator, null-propagation operator) is a binary operator that returns null if its first argument is null; otherwise it performs a dereferencing operation as specified by the second argument (typically an object member access, array index, or lambda invocation).

Before looking at how this operator works in PTS, let's see why we need it in the first place.

Suppose we want to get the phone number of an employee's manager by first getting the employee's department, then the manager of the department, and finally the manager's phone number.

If none of the objects in the chain can be null (i.e. they are all non-nullable), then the code might look like this in Java:

final String phoneNumber = employee.getDepartment().getManager().getPhoneNumber();

Here is the corresponding PTS code:

const phone_number = employee.department.manager.phone_number

Now let's suppose that later on the data model is changed, and every object in the chain (i.e. employee, department and manager) can now be null (i.e. they change from non-nullable to nullable). In that case:

  • The above Java code still compiles, but a NullPointerException is thrown at run-time if any object in the chain is null.

  • The above PTS code doesn't compile anymore, due to the built-in null-safety. There is no risk of a null pointer error at run-time.

The Java code could be fixed as follows:

String phoneNumber = null;
if ( employee != null ) {
    Department department = employee.getDepartment();
    if ( department != null ) {
        Manager manager = department.getManager();
        if ( manager != null ) {
            phoneNumber = manager.getPhoneNumber();
        }
    }
}

The PTS code also mutates into an if-monster, and phone_number can't be a constant anymore, it must be a variable:

variable phone_number = null
if employee is not null then
    const department = employee.department
    if department is not null then
        const manager = department.manager
        if manager is not null then
            phone_number = manager.phone_number
        .
    .
.

This PTS code can be simplified by using the safe navigation operator (?), followed by null. The code is now a one-liner again:

const phone_number = employee?null.department?null.manager?null.phone_number

Instead of writing ?null, we can also simply write ?:

const phone_number = employee?.department?.manager?.phone_number

This code is semantically equivalent to its verbose counterpart that uses nested if statements. The evaluation of the compound expression is aborted as soon as null is encountered in the chain. The inferred type for constant phone_number is string or null.

In PTS, usage of the ? operator is not restricted to type null — it may be used with any type (e.g. ?error, ?string, ?list<string>).

If no type is specified, the type defaults to null, since that's the most frequent use case. Therefore we can simply write ? instead of ?⁠null.

Multiple ? operators can be chained, to check for different types. For example, if the intermediate values in the chain are retrieved from functions that can return null or an error, we could write:

const phone_number = get_employee()?null?error \
    .get_department()?null?error \
    .get_manager()?null?error \
    .get_phone_number()

Now the evaluation of the expression is aborted as soon as an intermediate result evaluates to type null or error. The inferred type for phone_number is string or null or error.

Operator if_is

The if_is operator provides one of two possible values, depending on the type of an expression. Thus, if_is is used to execute a ternary operation involving a type.

Its syntax is as follows:

<expression_1> "if_is" <type> ":" <expression_2>

If <expression_1> evaluates to an instance of type <type>, then the result is <expression_2>, else the result is <expression_1>.

Operator if_is is often useful to provide a default value when an expression evaluates to null.

Suppose that product.comment is of type string or null. Now consider the following code that assigns a string to constant text:

const text = if product.comment is not null then product.comment else "No comment"

The code can be simplified by using the if_is operator:

const text = product.comment if_is null : "No comment"

This statement is semantically equivalent to the first statement that uses an if then else expression. If product.comment evaluates to a string, then that string is assigned to text. If product.comment evaluates to null, then the string "No comment" is assigned to text.

Instead of if_is null : the shorthand if_null : (or if_null:) can be used:

const text = product.comment if_null: "No comment"

Note

if_null: is a binary operator that works like the null coalescing operator in other languages.

For example, C# and JavaScript support the null coalescing operator, using the symbol ??. Here is a C# example shown on Wikipedia:

string pageTitle = suppliedTitle ?? "Default Title";

In PTS this would be written as:

const page_title = supplied_title if_null: "Default Title"

Besides the obvious benefit of succinct code, using if_null provides other advantages:

  • Code duplication is eliminated, since product.comment appears twice in the statement using an if expression, but only once in the code using if_null.

  • The code executes faster, because product.comment is only evaluated once.

  • The code can't end up in nasty (and sometimes very difficult to debug) run-time errors that can occur if product.comment is mutable.

    Consider the first version:

    const text = if product.comment is not null then product.comment else "No comment"

    Imagine a multithreaded application where product.comment is mutable, and the first evaluation of product.comment results in a string, while the second evaluation results in null because another thread has changed its value between the two evaluations. In that case null would erroneously be assigned to text.

    A problem like this (aka a race condition) is typically very unlikely to happen. Most likely, it never happens during development/tests. But later (maybe much later), when the code is executed millions of times a day in production, the likeliness increases, and suddenly an incredibly nasty bug appears randomly — very difficult to debug.

Note

An experienced and diligent developer, well aware of the potential problems with the if expression, would eliminate the three problems mentioned above (code duplication, performance, and risk of a race condition) by writing:

const comment = product.comment
const text = if comment is not null then comment else "No comment"

If product.comment is mutable, then a diligent compiler, designed to detect potential race conditions, generates an error (or at least a warning), because of the double occurrence of product.comment in the if expression.

And a diligent IDE would suggest to convert the code using an if expression into idiomatic PTS code which is succinct, fast, and reliable:

const text = product.comment if_null: "No comment"

Operator if_is can be used with any type, not just with null:

const object_displayed = object if_is password: "secret"

Several if_is operators can be chained. This is useful, for example, to stop the evaluation of an expression as soon as a value of a given type is encountered.

Consider an application to create digital documents. Suppose that the font used to render the document is determined in a cascading fashion: If the font is explicitly defined by an option in the document, then that font is used. If no font is defined in the document, then the application looks for a font defined in a shared config file. If the config file doesn't specify a font, then a hard-coded default font is used as fallback.

Without the if_is operator we would need to write verbose PTS code like this:

fn get_font ( context ) -> font

    variable font = context.document_font
    if font is not null then
        return font
    .
    
    font = context.config_font
    if font is not null then
        return font
    .

    return context.default_font
}

The if_is operator simplifies the code:

fn get_font ( context ) -> font =
    context.document_font if_null: context.config_font if_null: context.default_font

Helpful Statements

if ... is null Statement

In section Operator is we already saw how the is operator may be used in an if statement. Here is a reiteration of a previous example:

if customer.email_address is not null then
    send_email ( email_address )
else
    write_line ( "No email address!" )
.

case type of Statement

The case type of statement has already been introduced in section case type of Statement of the previous PTS article, titled Union Types in the Practical Type System.

Here is a reiteration of an example shown in that section:

case type of read_text_file ( file_path.create ( "example.txt" ) )
    
    is string as text // the string is stored in constant 'text'
        write_line ( "Content of file:" )
        write_line ( text ) // the previously defined constant 'text' is now used
    
    is null
        write_line ( "The file is empty." )
    
    is file_error as error
        write_line ( """The following error occurred: {{error.message}}""" )
.

As you can see, the second branch (is null) is executed when function read_text_file returns null.

Note

Besides a case type of statement, PTS also provides a case type of expression:

const message = case type of read_text_file ( file_path.create ( "example.txt" ) ) \
    is string: "text" \
    is null: "no text" \
    is error: "file read error"
write_line ( "Result: " + message )            

assert Statement

Sometimes we know more than the compiler does. For example we might know that a function declared to return a value of type string or null will never return null in a given context.

In such cases we can use an assert statement to express our assumption:

const result string or null = get_string_or_null ( ... )
assert result is not null
...
const size = result.size // valid because 'result' has been asserted to be non-null

In the above code, assert is used to state that the value stored in result will never be null.

As we all know, "to err is human". Therefore the assumption result is not null is checked at run-time, and an error is generated if the assumption turns out to be wrong.

Besides asserting not null we can also assert null, or assert any other type for a given expression:

assert problem is null
assert names is list<string>
assert result is not error

As in other languages, the assert keyword is followed by any boolean expression. Hence, in addition to asserting the type of an expression, assert can be used to express a wide range of assumptions. It may be used to assert object/state conditions, loop invariants, or any other conditions that are helpful to reliably document the code, simplify it, or optimize its performance.

Note

assert statements must always be free of side-effects, especially if a compiler flag allows to disable them for better performance.

We can provide a specific error message to be displayed at run-time, using the error_message: property at the end of the statement. Here are a few examples:

assert employee.name.size <= 50
assert customer.phone_number.starts_with ( "+" ) \
    error_message: "Phone number doesn't start with '+'"
assert index >= 1 and index <= list.size \
    error_message: """Index ({{index}}) out of bounds (1..{{list.size}})"""

Clause on

Clause on is used to execute a return or throw statement if an expressions evaluates to a specified type.

The general syntax of the on clause is as follows:

<expression> "on" <type> ( "as" <identifier> ) ? ":" <return_or_throw_statement>

If <expression> matches <type> then <return_or_throw_statement> is executed. The optional <identifier> can be used to store the result of <expression> in a constant which can then be used in <return_or_throw_statement> (see examples below).

Let's see why this is useful.

Here's an example of a recurring code pattern:

const value = get_value_or_null()
if value is null then
    return null
.

We can use the on clause to write semantically equivalent, but shorter code:

const value = get_value_or_null() on null : return null

Besides a return statement, a throw statement can be used too, to abort program execution:

const value = get_value_or_null() on null : throw application_error.create (
    "Unexpected 'null' returned by 'get_value_or_null'." )

Any type can be used in an on clause, and several on clauses can be chained:

const value = get_value_or_null_or_error() \
    on null : return null \
    on error as e : return e

on null : return null and on error as e : return e are both used frequently in practice. Therefore, the shorthands ^on_null and ^on_error can be used instead. The above code becomes:

const value = get_value_or_null_or_error() ^on_null ^on_error

... which is also semantically equivalent to the following verbose code:

const value = get_value_or_null_or_error()
if value is null then
    return null
.
if value is error then
    return value
.

The on clause may be used at the end of a constant assignment, variable assignment, or function call:

// constant assignment
const c number = get_string_or_number() on string : return null

// variable assignment
variable v = get_string_or_number() on string : return null
v = get_number_or_null() ^null

// function call (return value is null or an error)
close_connections() ^error

Flow-Sensitive Typing

Flow-sensitive typing (also called flow typing or occurrence typing) means that the compiler tracks and dynamically changes the type of object references (constants, variables, etc.), depending on where they are accessed in the code.

Flow typing was already briefly introduced in section Operator is of the previous article, titled Union Types in the Practical Type System.

In the context of null-handling, flow typing is convenient since it reduces the number of null checks required in the code, which leads to smaller and faster code.

For example, consider the following code (where size is a method of type string):

variable name string or null = null
variable size = name.size // invalid
name = "Bob"
size = name.size // valid

The second line is clearly invalid, and therefore rejected by the compiler. But the last line is valid, because at this time the value "Bob" is stored in name. However, without flow typing the compiler would generate an error, since the type of name is declared to be string or null. Flow typing eliminates this false positive, because the compiler tracks the values assigned to name, dynamically changes its type, and concludes that the last statement is valid.

Now consider this example, involving control flow:

variable name string or null = null
if foo() then
    name = "Bob"
else
    name = "Alice"
.
const size = name.size // valid

The last line is valid, because a string is assigned in both the then and else branches.

Now let's add some complexity to the previous code:

variable name string or null = null
if foo() then
    if bar() then
        name = "Bob"
    .
else
    name = "Alice"
.
const size = name.size // invalid

Now the last line isn't valid anymore, because if foo() returns true, and bar() returns false then name will be null. A compile-time error is generated.

To implement flow typing, the compiler analyzes all execution paths in the source code (considering all kinds of control flow statements, such as if, case, while, return, throw), and adapts the types of all object references in scope for each relevant location in the source code.

To cover flow typing everywhere in the code, the compiler must also analyze execution paths within expressions (besides analyzing statements).

Consider the following code:

const name string or null = ...
if ( name is not null and name.size > 50 )
    write_line ( "Name too long" )
.

The above code is valid, because when name.size is evaluated, name is guaranteed to be non-null by the preceding name is not null check.

If we accidentally inverted the order of the checks, or used an or instead of an and, the code wouldn't compile anymore:

if ( name.size > 50 and name is not null ) // compile-time error

if ( name is not null or name.size > 50 )  // compile-time error

Summary

PTS uses null to represent the absence of a value.

Union types are used to declare nullable object references (e.g. string or null).

Null-safety is natively built into the type system, therefore null pointer errors cannot occur.

PTS provides a set of dedicated operators and statements to facilitate null-handling as much as possible.

Flow typing reduces the number of null checks required in the code, which leads to smaller and faster code.

A Closing Note on null

There is no need to hate or fear null anymore.

null is a very useful concept, frequently used in many software applications.

The invention of null was not a "billion dollar mistake" per se. The mistake was the lack of type systems that ensure null-safety, as well as the lack of language features that make null-handling easy, safe, and enjoyable.

Acknowledgment

Many thanks to Tristano Ajmone for his useful feedback to improve this article.