Must() Be Misunderstood, in Go

In Go, a practice has developed around creating functions that perform a certain operation and panic if that operation fails. These functions are often prefixed with “Must” to signal that they will cause the program to crash if they cannot perform their intended function.

Generally, Must functions which panic should be avoided in Go beyond your application’s entry point. One case where they are very useful, and rightfully so, is when they are used during your application’s initialization, before requests are being served. That way, if something goes wrong, the application will not start, and you would be able to fix the issue before it starts serving requests.

An example of this is the MustCompile function in the regexp package. This function takes a regular expression string and returns a compiled regular expression. If the regular expression string is invalid, the function will panic.

// MustCompile is like Compile but panics if the expression cannot be parsed.
// It simplifies safe initialization of global variables holding compiled regular
// expressions.
func MustCompile(str string) *Regexp {
	regexp, err := Compile(str)
	if err != nil {
		panic(`regexp: Compile(` + quote(str) + `): ` + err.Error())
	}
	return regexp
}

Misusing Must Functions

The problem is Must functions are often misused and misunderstood. Here is how.

Using Must in user-serving request handlers

In a microservice handling incoming requests, panic() and MustXXX() functions should really only be used in the initialization code. If you use them in the request handlers, you will end up crashing the whole service, which is not what you want. Instead, you should return an error and let the caller decide what to do with it.

func NewService() *Service {
    // ...
    // You can use Must functions here during the bootstrapping phase of your application.
    db := MustGetPostgresql()
    // The app will crash if the database connection fails, which is fine because without database, 
    // the app cannot fulfil its purpose.
}

func (s *Service) HandleRequest(w http.ResponseWriter, r *http.Request) {
    // ...
    // do not use Must here
    // return an error instead
    err := s.db.SaveUser(r.Context(), user)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
}

Using Must in libraries

In a library, you should never use Must functions, at least not exclusively. Instead, you should return an error and let the caller decide what to do with it. For instance, you can expose two functions, one that returns an error and one that panics. That way, the caller can choose which one to use.

func DoSomething() error {
    // ...
}

func MustDoSomething() {
    err := DoSomething()
    if err != nil {
        panic(err)
    }
}

Misjudging the purpose

Must functions are mainly intended for situations where if an error occurs, it is not recoverable, and the only sensible action is to crash the program. They are commonly used during initialization (like loading essential configurations or templates). If an error occurs in a Must function during the normal operation of a program, it will lead to crashes, which can be very disruptive.

func main() {
    // Attempt to load a template file. If the file doesn't exist, our program cannot proceed.
    // Using a 'Must' function here is appropriate because it's part of the initialization.
    // If the template is not loaded, there's nothing the program can do.
    tmpl := must(template.ParseFiles("templates/base.html"))

    // The rest of the program logic would go here, for example, starting a web server,
    // processing input, etc., where you should avoid using 'Must' functions to
    // ensure the program can handle errors gracefully without crashing unexpectedly.
}

// must is a wrapper function for operations that we assume should never fail during
// the normal operation of our software. If the passed-in operation results in an error,
// the function will panic and cause the program to crash.
func must(t *template.Template, err error) *template.Template {
    if err != nil {
        // Instead of handling the error, we panic, as we've judged this to be a
        // critical, unrecoverable error. In a real-world application, it's crucial
        // to ensure this assumption is correct, as it leads to program termination.
        panic(err)
    }
    return t
}

Overusing Must functions

Some developers overuse Must functions for the sake of convenience, as these functions do not return an error that needs to be handled. This practice can lead to fragile code that panics unexpectedly in situations where an error could have been recovered gracefully.

func main() {
    // Imagine a scenario where we are creating a new file.
    // Using a 'Must' function here for convenience, i.e., avoiding writing error handling.
    must(os.Create("example.txt"))

    // Further down in the code, we try to do more file operations or other work
    // that is not critical for application survival. Still, we continue to use
    // 'Must' functions out of habit or for convenience.

    // Trying to delete a non-critical temporary file. This operation could fail for
    // various reasons (e.g., file does not exist), but it's not a good use of 'Must'
    // because it's not a catastrophic failure if we can't delete a temporary file.
    // The program could continue to work correctly, but it will crash instead.
    must(os.Remove("temp.txt"))

    // The overuse of 'Must' functions for non-critical operations means the program
    // is much more likely to crash unnecessarily.
}

// must is a helper that panics on any error. This is an example of what NOT to do for
// regular operations, as it can cause the program to crash if any little thing goes wrong,
// even if the error was recoverable or insignificant to the program's overall functionality.
func must(_ interface{}, err error) {
    if err != nil {
        panic(err) // Forces a program crash over an error that might have been non-critical.
    }
}

// Correct practice would be to handle errors with regular error checking and only use 'Must'
// patterns for initialization operations that, if they fail, leave the application non-functional.

Confusing panic with error handling

Go’s error handling is designed to be explicit, with errors as values that should be returned and checked. Using Must functions confuses this pattern, as it introduces panic as a form of error handling, which it’s not intended for. Panicking is more akin to a runtime exception in other languages and should be reserved for exceptional, unrecoverable errors, that leave the program in an unknown or unstable state.

Testing challenges

Functions that panic are more difficult to test because they can cause the test runner to stop. Developers need to use recovery mechanisms to catch the panic in the testing code, which can make test suites more complicated and harder to understand.

Indeed, normally, tests are designed to expect certain outcomes or errors, which they handle gracefully to produce informative output. However, a panic caused by a Must function isn’t a normal error but a catastrophic event that typically terminates the application. If a panic occurs during a test, it can stop the test runner entirely, preventing other tests from running and disrupting the understanding of what is happening within the test suite.

If you indeed want to check a function, given specific inputs, should panic, you can use testify/assert.Panics() and testify/assert.NotPanics().

func TestMustDoSomething(t *testing.T) {
    assert.Panics(t, func() {
        MustDoSomething(badInput)
    })
    assert.NotPanics(t, func() {
        MustDoSomething(goodInput)
    })
}

Concurrency considerations

When writing concurrent code, using functions that may panic requires careful consideration. If a goroutine panics, it’s necessary to handle the panic appropriately to avoid unexpected behavior or crashes. Unhandled panics in goroutines can terminate the whole program.

package main

import (
	"fmt"
	"sync"
	"time"
)

func main() {
	var wg sync.WaitGroup

	// Starting multiple goroutines
	for i := 0; i < 5; i++ {
		wg.Add(1)
		go func(i int) {
			defer wg.Done() // Ensure goroutine signals it's done when finished.
			DoRiskyTask(i)  // This function might panic depending on the input.
		}(i)
	}

	wg.Wait() // Wait for all goroutines to complete.

	fmt.Println("Main program exited normally.")
}

// DoRiskyTask is a contrived function that panics if it receives a specific value.
func DoRiskyTask(i int) {
	// If i is 2, we simulate a panic (maybe a rare, unexpected condition in real-world scenarios)
	if i == 2 {
		panic(fmt.Sprintf("simulated panic for value: %d", i))
	}

	// Simulate work that takes time.
	time.Sleep(1 * time.Second)
	fmt.Printf("Task %d finished successfully.\n", i)
}

Without recover(), you will see:

$ go run main.go
panic: simulated panic for value: 2

So all the other goroutines are terminated, and the program exits. Which may not be what you want, as you may want to continue processing the other goroutines, or at least exit them gracefully.

Ignoring error context

When a Must function panics, it’s often with a generic message, and the original error is not always included. This practice can lose the context and make it harder to diagnose the issue because the original error may contain valuable information.

Misunderstanding the impact on performance

While it’s not the primary concern with Must functions, it’s worth noting that introducing panic/recover mechanisms can have The guys at DoltHub have done some benchmarking and found that using panic/recover does not have a significant impact on performance. They actually showed it can be faster than returning an error up the function stack to the entry point.

So When To Use Must, Then?

Without trying to be too prescriptive, here are some guidelines to help you decide when to use Must functions. Most scenarios are by convention and they may different from one project to another.

Yes, there are certain scenarios where using Must functions or similar panic-causing constructs is appropriate and even recommended. These are typically situations where failure of the function indicates a non-recoverable problem that necessitates immediate program termination. Here are some instances:

Application Initialization

During the startup phase of an application, you might use Must functions to load critical resources or initialize settings without which the application cannot run. Examples include loading configuration files, parsing essential templates, or establishing database connections. If these operations fail, the application cannot continue to start, and panicking provides a clear, immediate signal of the startup failure.

config := mustLoadConfig("config.yaml") // If the configuration cannot be loaded, the application cannot run.

Assertion During Development

Must functions can be useful as a development tool to assert conditions that you believe should never fail. They act as safeguards during the development phase, causing the application to panic and drawing attention to the issue that needs immediate resolution. It’s a way to enforce that the program is operating under the expected conditions and configurations.

mustParseURL("https://example.com") // During development, you're sure this URL should never fail to parse.

You do need to remember to remove these assertions before deploying the application to production, as they can cause unexpected crashes if they are not removed.

Testing Invariants

In testing scenarios, you might use Must functions to ensure that test setup conditions are met. If a Must function fails during test setup, it indicates that something is fundamentally wrong with the test environment, and it cannot provide valid testing results.

t := template.Must(template.New("test").Parse("Hello, {{.Name}}")) // If the template can't parse, the test environment is compromised.

Standard Library Initialization

Go’s standard library often uses Must functions to initialize global variables, particularly with package-level regular expressions or templates. The rationale is that these are set at the package level and are considered essential for the package’s functionality.

var validID = regexp.MustCompile(`^[a-z]+[[0-9]+]$`) // If this regexp is invalid, the package is fundamentally broken.

In these scenarios, the common theme is that a failure in the Must function represents a severe, non-recoverable error that justifies terminating the program.

The Must functions are not used for regular error-handling pathways but are reserved for these exceptional situations. However, overuse or misuse of Must functions outside of these scenarios can lead to fragile code and applications that crash unexpectedly, which is undesirable in production environments.