Null-Safety in the Practical Type System (PTS)
|
First Published |
2023-12-11 |
|
Author |
Christian Neumanns |
|
Editor |
Tristano Ajmone |
|
License |
Note
This is part 5 in a series of articles titled How to Design a Practical Type System to Maximize Reliability, Maintainability, and Productivity in Software Development Projects.
It is recommended (but not required for experienced programmers) to read the articles in their order of publication, starting with Part 1: What? Why? How?.
Please be aware that PTS is a new, not-yet-implemented paradigm. As explained in section History of the article Essence and Foundation of the Practical Type System (PTS), PTS has been implemented in a proof-of-concept project, but a public PTS implementation isn't available yet — you can't try out the PTS source code examples shown in this article.
For a quick summary of previous articles you can read Summary of the Practical Type System (PTS) Article Series.
Introduction
The absence of a value is among the most important concepts a type system has to deal with — in one way or another.
Consider the scenario where the delivery date of a given order is still unknown. In this case, we can't assign a date to delivery_date, since there is no date available. We have to deal with the absence of a value.
Most software applications have to deal with many similar situations where no value is available for some object references.
A practical type system should therefore provide first class support to handle all cases gracefully. Handling the absence of a value should be easy, reliable, and maintainable.
This article explains how PTS aims to achieve this.
Approaches to Handle the "Absence of a Value"
Before showing how PTS handles the absence of a value, let's first look at common approaches.
Note
Readers only interested in the PTS approach can skip the next section.
Common Approaches
As far as I know, there are three common approaches to handle the absence of a value.
-
nullMany programming languages use the symbol
nullto represent the absence of a value. Instead ofnull, some languages usenil,void,nothing,missing, etc., but the basic idea is the same.Note
For a basic introduction to
nullyou can read my article A Quick and Thorough Guide tonull. -
Null-Object Pattern
In some languages there's no native support for handling the absence of a value.
In such environments the null object pattern might be used. Wikipedia states:
Instead of using a null reference to convey absence of an object (for instance, a non-existent customer), one uses an object which implements the expected interface, but whose method body is empty. A key purpose of using a null object is to avoid conditionals of different kinds, resulting in code that is more focused, quicker to read and follow - i e [sic] improved readability. One advantage of this approach over a working default implementation is that a null object is very predictable and has no side effects: it does nothing.
Simple examples of applying the null object pattern would be to use zero for numbers, an empty string for object references of type string, and an empty collection for collection types.
At first, this might seem to be a good solution, because not using
nullalso means to get rid of the dreaded null pointer error.However, it turns out that the null object pattern creates more problems than it solves, and renders debugging more difficult — it's a poor man's solution. For more information and examples, you can read sections Using Zero Instead of Null and The Null Object Pattern in my article Why We Should Love 'null'.
-
Option/MaybetypeThis approach is based on the following premises:
-
nullis not supported in the type system. -
An
Optiontype is used to handle the absence of a value. This type is also namedMaybe,Optional, etc., but the basic idea is the same.
One can think of
Option/Maybeas a container that is either empty or contains a value.Most
Option/Maybeimplementations also provide specific functions/methods that are useful in the context of this type.In some languages the
Option/Maybetype is also a monad. For example, Haskell provides theMaybemonad.Note
For an introduction to monads (tailored to programmers who are unfamiliar with functional languages) you can read my article Simple Introduction to Monads — With Java Examples.
-
PTS Approach
PTS uses null. Yes, null!
This might come as a surprise, because modern programming languages tend to adopt the Option/Maybe approach.
Discussing the pros and cons of null vs Option/Maybe is beyond the scope of this article, but for an in-depth discussion you can read my article Null-Safety vs Maybe/Option — A Thorough Comparison. In a nutshell, that article shows:
-
Null-handling (if well implemented) is more convenient and practical for software developers than
Option/Maybe, but it's more challenging to implement as a native feature in the language. -
The
Option/Maybetype is relatively easy to implement in the standard library.
If a choice must be made between easing the life of application developers or that of language engineers and compiler developers then, in the context of PTS, the preference goes to application developers. Therefore embracing null is a better approach in PTS.
Further reasons for choosing null have already been revealed in the previous article, titled Union Types in the Practical Type System.
However, null is a viable approach only if null-safety is guaranteed, as explained in the following section.
Why Do We Need Null-Safety?
History shows that the most frequent bug in many software applications is the infamous null pointer error, nowadays synonym with "the billion dollar mistake", a term coined by Sir Tony Hoare, Professor Emeritus, the inventor of null.
Professor John Sargeant from the Department of Computer Science, University of Manchester, puts it like this in his article Null pointer exceptions:
Of the things which can go wrong at runtime in Java programs, null pointer exceptions are by far the most common.
— Prof. John Sargeant
Here is an example of a null-pointer exception in Java:
String name = null;
int length = name.length();
Running this code will report the following run-time error, because method length() can't be executed on a null object:
Exception in thread "main" java.lang.NullPointerException: Cannot invoke "String.length()" because "name" is null ...
After reporting the error, the application aborts immediately. The consequences in production are unpredictable — they vary from harmless to disastrous.
Besides Java, other popular programming languages are also vulnerable to null pointer errors, for example C, C++, Go, JavaScript, Python, and Ruby. While modern C# has non-null types and specific null-checking features (since version 8), it still allows the use of null references, which can lead to null pointer errors.
Practice shows:
Many software applications written in non-null-safe languages are
copiously filled with null pointer bugs
lurking in the code and
waiting to hit at some unpredictable time in the future
causing undefined outcomes
ranging from harmless to very expensive disasters.
We need to eradicate the null pointer error.
We need null-safety.
Think about it: Null-safety eradicates the most common bug in languages that support null. Therefore null-safety is a most welcome feature — a crucial milestone in our pursuit of enhanced reliability, maintainability, and productivity.
Let's see how null-safety is achieved in PTS.
How Does It Work?
Fundamental Principles
PTS null-safety is based on the following principles:
-
PTS provides a built-in type dedicated to represent the absence of a value. This type is named
nulland it has a single value:null. -
While
nullis a valid value for typenull, it is invalid for all other types. In other words, all other types are non-null.For example, a function with an input parameter of type
stringcannot be called withnullas input. -
Union types are used to declare nullable object references (as covered already in the previous article titled Union Types in the Practical Type System).
nullcan be assigned to a nullable object reference.For example, a function with an input parameter of type
string or nullcan be called withnullas input. -
The compiler ensures that a method call on an object reference is only valid if the object reference is guaranteed to be non-null at run-time. Thus, null pointer errors cannot occur. The type system is null-safe.
For example,
foo.bar()is only valid iffoois guaranteed to be non-null at run-time. -
A dedicated set of operators and statements facilitates null-handling and the development of null-safe code.
These are the fundamental design principles.
Now let's dig a bit deeper.
The null Type
Type null is a built-in type.
This type has a single value, also called null, in its set of allowed values. Thus the cardinality of type null is one.
null is used to represent the absence of a value.
For example, the assignment:
delivery_date = null
... states the absence of a date for delivery_date. No date is assigned to delivery_date.
Note
The code delivery_date = null doesn't tell us the reason for delivery_date pointing to null. The reason might be that a delivery date is not yet available. But the reason might also be that the delivery date has not yet been entered into the database. null just means there is no value available. It doesn't mean anything else, unless a specific meaning has been explicitly specified in a given context.
Type Hierarchy
The following class diagram shows the top types in the PTS type hierarchy:
Note
The convention in class diagrams is to style non-instantiable/abstract types (explained later) in italic.
At the top of the hierarchy sits type any — the root of all types in PTS. Types non_null and null both inherit from any. All other types inherit from non_null.
Note
Type inheritance has been briefly introduced in section Type Inheritance of a previous PTS article, titled Record Types in the Practical Type System. This topic isn't covered here — it's assumed that readers of this article are familiar with the concept of type inheritance.
Now let's have a closer look at these types.
Type any
The set of allowed values for the root type any is empty — its cardinality is zero. Simply put, type any doesn't have a value.
This implies that it's impossible to create an object of type any. Objects can only be created for some child-types/descendants of any. Type any is therefore called a non-instantiable (or abstract) type, and styled in italic in the above diagram.
Moreover, nobody is allowed to define child-types of any. Type any is a so-called sealed type. Its child-types are fixed: non_null and null.
The PTS source code for any looks like this:
type any \
factories: none \ (1)
child_types: non_null, null (2)
.
(1) Instances of any can't be created.
(2) non_null and null are the only child-types.
Type non_null
Type non_null is a child of type any.
The set of allowed values for type non_null is empty too — its cardinality is zero. Simply put, type non_null doesn't have a value.
Like type any, type non_null is a non-instantiable type too. It's also a sealed type, as will be explained in the next PTS article.
Type null
Type null is a child of type any.
As seen already in a previous section, type null has a single value, called null, in its set of allowed values. The cardinality of type null is one. Simply put, type null has only one value: null.
Unlike types any and non_null, type null is an instantiable (or concrete) type.
Like any and non_null, null is also a sealed type. There are no child-types of null, and nobody is allowed to create child-types.
Other Types
All remaining built-in types not shown in the diagram (string, number, list, etc.), as well as all user-defined types (e.g. customer, supplier, product), inherit from non_null.
Rules
According to the important Liskov substitution principle, any type in a type inheritance tree is compatible with all its parent types.
Applying this principle to the above type inheritance tree implies the following rules:
-
Any value of any type can be assigned to an object reference declared to be of type
any.For example, a function with an input parameter of type
anycan be called with the following values:null,"foo",123, or any other value of any other type. -
nullis an invalid value for object references of typenon_null(or any of its descendants).For example:
-
A function with an input parameter of type
non_nullcannot be called withnullas input. But any value of every other type ("foo",123, etc.) is allowed as input. -
A function with an input parameter type that's a descendant of
non_null(e.g.string,customer) cannot be called withnullas input.Note
This is exactly the opposite of what's allowed in many popular languages, such as C, Java, JavaScript, Python, Ruby, where every object reference can be
null. In Java, for example,nullcan be assigned to an input parameter of typeString.
-
-
It doesn't make sense to declare an object reference of type
null. Therefore the compiler doesn't allow it.For example, a function input parameter cannot be of type
null.Type
nullcan only be used as a member of a union type, as explained in the next section.
Nullable Object References
A union type (see Union Types in the Practical Type System) is used to declare a nullable object reference. An object reference is nullable if its type is a union type that contains member null (e.g. string or null). In that case the value null is valid.
For example, if function foo has an input parameter of type string or null then calling it with null as input is valid, as shown in the following code:
function foo ( string or null )
// body
.
foo ( "bar" ) // ok
foo ( null ) // ok
On the other hand, null is not allowed if the input parameter type is simply string:
function foo ( string )
// body
.
foo ( "bar" ) // ok
foo ( null ) // compile-time error
Helpful Operators
In this section we'll have a look at useful operators that facilitate null-handling.
Operator is
Note
The is operator has already been introduced in section Operator is of the previous PTS article titled Union Types in the Practical Type System.
This section only covers its usage in the context of null-handling.
The is operator checks whether an expression is of a given type. The result is a boolean value. For example, "foo" is string evaluates to true, while 123 is string evaluates to false.
Hence, this operator is useful to check whether a given expression evaluates to null. Here is an example:
if customer.email_address is null then
write_line ( "No email address!" )
else
send_email ( email_address )
.
Instead of is, we can use is not to invert the check. Hence the above code can also be written like this:
if customer.email_address is not null then
send_email ( email_address )
else
write_line ( "No email address!" )
.
Besides using is in an if statement, we can also use it in an if expression:
const has_email = if customer.email_address is null then "no" else "yes"
write_line ( """Customer has email: {{has_email}}""" )
Operator if_is
The if_is operator provides one of two possible values, depending on the type of an expression. Thus, if_is is used to execute a ternary operation involving a type.
Its syntax is as follows:
<expression_1> "if_is" <type> ":" <expression_2>
If <expression_1> evaluates to an instance of type <type>, then the result is <expression_2>, else the result is <expression_1>.
Operator if_is is often useful to provide a default value when an expression evaluates to null.
Suppose that product.comment is of type string or null. Now consider the following code that assigns a string to constant text:
const text = if product.comment is not null then product.comment else "No comment"
The code can be simplified by using the if_is operator:
const text = product.comment if_is null : "No comment"
This statement is semantically equivalent to the first statement that uses an if then else expression. If product.comment evaluates to a string, then that string is assigned to text. If product.comment evaluates to null, then the string "No comment" is assigned to text.
Instead of if_is null : the shorthand if_null : (or if_null:) can be used:
const text = product.comment if_null: "No comment"
Note
if_null: is a binary operator that works like the null coalescing operator in other languages.
For example, C# and JavaScript support the null coalescing operator, using the symbol ??. Here is a C# example shown on Wikipedia:
string pageTitle = suppliedTitle ?? "Default Title";
In PTS this would be written as:
const page_title = supplied_title if_null: "Default Title"
Besides the obvious benefit of succinct code, using if_null provides other advantages:
-
Code duplication is eliminated, since
product.commentappears twice in the statement using anifexpression, but only once in the code usingif_null. -
The code executes faster, because
product.commentis only evaluated once. -
The code can't end up in nasty (and sometimes very difficult to debug) run-time errors that can occur if
product.commentis mutable.Consider the first version:
const text = if product.comment is not null then product.comment else "No comment"Imagine a multithreaded application where
product.commentis mutable, and the first evaluation ofproduct.commentresults in astring, while the second evaluation results innullbecause another thread has changed its value between the two evaluations. In that casenullwould erroneously be assigned totext.A problem like this (aka a race condition) is typically very unlikely to happen. Most likely, it never happens during development/tests. But later (maybe much later), when the code is executed millions of times a day in production, the likeliness increases, and suddenly an incredibly nasty bug appears randomly — very difficult to debug.
Note
An experienced and diligent developer, well aware of the potential problems with the if expression, would eliminate the three problems mentioned above (code duplication, performance, and risk of a race condition) by writing:
const comment = product.comment
const text = if comment is not null then comment else "No comment"
If product.comment is mutable, then a diligent compiler, designed to detect potential race conditions, generates an error (or at least a warning), because of the double occurrence of product.comment in the if expression.
And a diligent IDE would suggest to convert the code using an if expression into idiomatic PTS code which is succinct, fast, and reliable:
const text = product.comment if_null: "No comment"
Operator if_is can be used with any type, not just with null:
const object_displayed = object if_is password: "secret"
Several if_is operators can be chained. This is useful, for example, to stop the evaluation of an expression as soon as a value of a given type is encountered.
Consider an application to create digital documents. Suppose that the font used to render the document is determined in a cascading fashion: If the font is explicitly defined by an option in the document, then that font is used. If no font is defined in the document, then the application looks for a font defined in a shared config file. If the config file doesn't specify a font, then a hard-coded default font is used as fallback.
Without the if_is operator we would need to write verbose PTS code like this:
fn get_font ( context ) -> font
variable font = context.document_font
if font is not null then
return font
.
font = context.config_font
if font is not null then
return font
.
return context.default_font
}
The if_is operator simplifies the code:
fn get_font ( context ) -> font =
context.document_font if_null: context.config_font if_null: context.default_font
Helpful Statements
if ... is null Statement
In section Operator is we already saw how the is operator may be used in an if statement. Here is a reiteration of a previous example:
if customer.email_address is not null then
send_email ( email_address )
else
write_line ( "No email address!" )
.
case type of Statement
The case type of statement has already been introduced in section case type of Statement of the previous PTS article, titled Union Types in the Practical Type System.
Here is a reiteration of an example shown in that section:
case type of read_text_file ( file_path.create ( "example.txt" ) )
is string as text // the string is stored in constant 'text'
write_line ( "Content of file:" )
write_line ( text ) // the previously defined constant 'text' is now used
is null
write_line ( "The file is empty." )
is file_error as error
write_line ( """The following error occurred: {{error.message}}""" )
.
As you can see, the second branch (is null) is executed when function read_text_file returns null.
Note
Besides a case type of statement, PTS also provides a case type of expression:
const message = case type of read_text_file ( file_path.create ( "example.txt" ) ) \
is string: "text" \
is null: "no text" \
is error: "file read error"
write_line ( "Result: " + message )
assert Statement
Sometimes we know more than the compiler does. For example we might know that a function declared to return a value of type string or null will never return null in a given context.
In such cases we can use an assert statement to express our assumption:
const result string or null = get_string_or_null ( ... )
assert result is not null
...
const size = result.size // valid because 'result' has been asserted to be non-null
In the above code, assert is used to state that the value stored in result will never be null.
As we all know, "to err is human". Therefore the assumption result is not null is checked at run-time, and an error is generated if the assumption turns out to be wrong.
Besides asserting not null we can also assert null, or assert any other type for a given expression:
assert problem is null
assert names is list<string>
assert result is not error
As in other languages, the assert keyword is followed by any boolean expression. Hence, in addition to asserting the type of an expression, assert can be used to express a wide range of assumptions. It may be used to assert object/state conditions, loop invariants, or any other conditions that are helpful to reliably document the code, simplify it, or optimize its performance.
Note
assert statements must always be free of side-effects, especially if a compiler flag allows to disable them for better performance.
We can provide a specific error message to be displayed at run-time, using the error_message: property at the end of the statement. Here are a few examples:
assert employee.name.size <= 50
assert customer.phone_number.starts_with ( "+" ) \
error_message: "Phone number doesn't start with '+'"
assert index >= 1 and index <= list.size \
error_message: """Index ({{index}}) out of bounds (1..{{list.size}})"""
Clause on
Clause on is used to execute a return or throw statement if an expressions evaluates to a specified type.
The general syntax of the on clause is as follows:
<expression> "on" <type> ( "as" <identifier> ) ? ":" <return_or_throw_statement>
If <expression> matches <type> then <return_or_throw_statement> is executed. The optional <identifier> can be used to store the result of <expression> in a constant which can then be used in <return_or_throw_statement> (see examples below).
Let's see why this is useful.
Here's an example of a recurring code pattern:
const value = get_value_or_null()
if value is null then
return null
.
We can use the on clause to write semantically equivalent, but shorter code:
const value = get_value_or_null() on null : return null
Besides a return statement, a throw statement can be used too, to abort program execution:
const value = get_value_or_null() on null : throw application_error.create (
"Unexpected 'null' returned by 'get_value_or_null'." )
Any type can be used in an on clause, and several on clauses can be chained:
const value = get_value_or_null_or_error() \
on null : return null \
on error as e : return e
on null : return null and on error as e : return e are both used frequently in practice. Therefore, the shorthands ^on_null and ^on_error can be used instead. The above code becomes:
const value = get_value_or_null_or_error() ^on_null ^on_error
... which is also semantically equivalent to the following verbose code:
const value = get_value_or_null_or_error()
if value is null then
return null
.
if value is error then
return value
.
The on clause may be used at the end of a constant assignment, variable assignment, or function call:
// constant assignment
const c number = get_string_or_number() on string : return null
// variable assignment
variable v = get_string_or_number() on string : return null
v = get_number_or_null() ^null
// function call (return value is null or an error)
close_connections() ^error
Flow-Sensitive Typing
Flow-sensitive typing (also called flow typing or occurrence typing) means that the compiler tracks and dynamically changes the type of object references (constants, variables, etc.), depending on where they are accessed in the code.
Flow typing was already briefly introduced in section Operator is of the previous article, titled Union Types in the Practical Type System.
In the context of null-handling, flow typing is convenient since it reduces the number of null checks required in the code, which leads to smaller and faster code.
For example, consider the following code (where size is a method of type string):
variable name string or null = null
variable size = name.size // invalid
name = "Bob"
size = name.size // valid
The second line is clearly invalid, and therefore rejected by the compiler. But the last line is valid, because at this time the value "Bob" is stored in name. However, without flow typing the compiler would generate an error, since the type of name is declared to be string or null. Flow typing eliminates this false positive, because the compiler tracks the values assigned to name, dynamically changes its type, and concludes that the last statement is valid.
Now consider this example, involving control flow:
variable name string or null = null
if foo() then
name = "Bob"
else
name = "Alice"
.
const size = name.size // valid
The last line is valid, because a string is assigned in both the then and else branches.
Now let's add some complexity to the previous code:
variable name string or null = null
if foo() then
if bar() then
name = "Bob"
.
else
name = "Alice"
.
const size = name.size // invalid
Now the last line isn't valid anymore, because if foo() returns true, and bar() returns false then name will be null. A compile-time error is generated.
To implement flow typing, the compiler analyzes all execution paths in the source code (considering all kinds of control flow statements, such as if, case, while, return, throw), and adapts the types of all object references in scope for each relevant location in the source code.
To cover flow typing everywhere in the code, the compiler must also analyze execution paths within expressions (besides analyzing statements).
Consider the following code:
const name string or null = ...
if ( name is not null and name.size > 50 )
write_line ( "Name too long" )
.
The above code is valid, because when name.size is evaluated, name is guaranteed to be non-null by the preceding name is not null check.
If we accidentally inverted the order of the checks, or used an or instead of an and, the code wouldn't compile anymore:
if ( name.size > 50 and name is not null ) // compile-time error
if ( name is not null or name.size > 50 ) // compile-time error
Summary
PTS uses null to represent the absence of a value.
Union types are used to declare nullable object references (e.g. string or null).
Null-safety is natively built into the type system, therefore null pointer errors cannot occur.
PTS provides a set of dedicated operators and statements to facilitate null-handling as much as possible.
Flow typing reduces the number of null checks required in the code, which leads to smaller and faster code.
A Closing Note on null
There is no need to hate or fear null anymore.
null is a very useful concept, frequently used in many software applications.
The invention of null was not a "billion dollar mistake" per se. The mistake was the lack of type systems that ensure null-safety, as well as the lack of language features that make null-handling easy, safe, and enjoyable.
Acknowledgment
Many thanks to Tristano Ajmone for his useful feedback to improve this article.