defer reference implementation for C

In the following code snippet we have one guarded block and three deferred statements.

guard {
  void * const p = malloc(25);
  if (!p) break;
  defer free(p);

  void * const q = malloc(25);
  if (!q) break;
  defer free(q);

  if (mtx_lock(&mut)==thrd_error) break;
  defer mtx_unlock(&mut);

  // all resources acquired
}

The idea is that we indicate with a defer keyword that the statement, e.g a call to free, is only to be executed at the end of the guarded block, and, that we want this action to happen unconditionally in which way ever the guarded block is left. For the three deferred statements together it says that they should be executed in the inverse order than they were encountered. So the control flow for this code example can be visualized as follows:

Here, the dashed lines represent circumstancial control flow that might arise when some resources are not available or when the execution is interrupted by a signal.

This new technique has at least two advantages to commonly used C or C++ techniques:

proximity

The cleanup code (free or mtx_unlock) is coded close to the place where its need arises.

visibility

The cleanup code is not hidden in some previously defined function (such as for atexit handlers) or constructor/destructor pairs (C++)

For normal control flow (without intermediate return, exit, …) code with similar properties can be coded with existing tools. The above is equivalent to something like the following

{
   void * const p = malloc(25);
   if (!p) goto DEFER0;
   if (false) {
     DEFER1:
       free(p);
       goto DEFER0;
   }

   void * const q = malloc(25);
   if (!q) goto DEFER1;

   if (false) {
     DEFER2:
       free(q);
       goto DEFER1;
   }

   if (mtx_lock(&mut)==thrd_error) goto DEFER2;

   if (false) {
     DEFER3:
       mtx_unlock(&mut);
       goto DEFER2;
   }

   // all resources acquired

   goto DEFER3;
   DEFER0:;
}

Here, the if(false) clauses guarantee that the deferred statements are jumped over when they are first met, and the labels and goto statements implement the hops from back to front to execute the deferred statements eventually.

Obviously, most C programmers would not code like this but they would prefer to write down a linearization of the above, which is a quite common idiom for cleanup handling in C:

{
   void * const p = malloc(25);
   if (!p) goto DEFER0;

   void*const q = malloc(25);
   if (!q) goto DEFER1;

   if (mtx_lock(&mut)==thrd_error) goto DEFER2;

   // all resources acquired

   mtx_unlock(&mut);

 DEFER2:
   free(q);
 DEFER1:
   free(p);
 DEFER0:;
}

This has the advantage of only making the circumstantial control flow explicit (with three *=goto=) but it does that for the price of proximity; the cleanup code is far from the place where its need arises.

Nevertheless, even this linearization needs some form of naming convention for the labels. For more complicated code the maintenance of these jumps can be tricky and prone to errors. This shows another advantage of the defer approach:

maintainability

The cleanup specification is not dependent on arbitrary naming such as labels (C) or RAII classes (C++) and does not change when defer or break statements are added or removed.

Another important property that is much more difficult to implement in C (and that needs try/catch blocks in C++) is that all exits from the guarded block are detected and acted upon: break, return, thrd_exit, exit, panic, or an interruption by a signal. That is, unless there are nasal deamons flying around, we have a forth important property

robustness

Any deferred statement is guaranteed to be executed eventually.

This is different from C++'s handling of destructors, which are only guaranteed to be executed if there is a try/catch block underneath.

This principle of deferred execution extends to nested guarded blocks in a natural way, even if they are stacked in different function calls.

In the example above the control flow that was the result of deferring statements was static, that is the defer statements themselves were not placed into conditionals or loops. Being able to place them inside an if, for example, gives the programmer more flexibility, in particular it allows to only defer a statement if the corresponding resource has really been needed.

// create a small local buffer as a compound literal
double* p = (len > MAXLEN) ? 0 : (double[MAXLEN]){ 0 };

// if len is too big, go to the heap, instead
if (p) {
   p = defer_calloc(len, sizeof double);
   defer free(p);
}
// now use p as a buffer until we leave the function

Here, the deferred statement will only be evaluated at the termination of the guarded block only in cases where the buffer referenced by p is dynamically allocated. To support such constructs, our implementation determines the order and number of times each deferred statement is executed at run-time. This approach requires allocating storage at run-time to maintain defer state. The defer mechanism provides fault tolerance by continuing to operate properly in the event of the failure of a defer statement by enduring the deferred statement is executed to release the resource.

defer_calloc and the defer statement may both exhaust memory. If the call succeeds (so a resource is allocated) but the defer mechanism fails, the call to free is evaluated immediately and the execution of the enclosing guarded block is terminated by a panic with an error code of DEFER_ENOMEM.

The current idea in our proposal to the C standardization is to have each function body to implement a guarded block, such that an explicit guard statement is not necessary for many common use cases. Generally, this can not be done by macro wonders but needs compiler magic.

The reference implementation achieves that for the gcc family of compilers by some internal magic by keeping track of the stack pointer. This has some caveats, see below. Nevertheless in this documentation we will always speak about a guarded block, regardless if a guard statement is given explicitly or if the function body acts as such.

There are three "severity" levels for termination of a guarded block, break or similar just terminates the guarded block, return terminates all guarded blocks of the same function and exit or panic of the same thread across all function calls.

A deferred statement can be triggered in two ways.

• explicitly: by leaving a guarded block when coming to the end of it or by using break, defer_break, exit … from within.

• implicitly: from a signal handler. Two signal handlers are provided defer_sig_flag and defer_sig_jump. Their actions are as their names indicate. The first just sets a flag and the user code then typically investigates that flag and acts accordingly. This would typically be used for a signal like SIGINT where the user interrupts the program execution. The second handler jumps to the first defer clause on the defer stack and triggers a panic. This one is more appropriate for signals that are related to faulty operations that by themselves are not recoverable.

  This implementation does not install any of these handlers by default.

Once execution starts inside a deferred statement, the condition that lead there can be investigated by means of recover. This returns an integer error code that indicates what happened. If it is 0, this execution of the defer clause is just "normal" retreat as of break, return or exit. If it is any other value something "remarkable" might have happened, generally a panic is on its way down in the call stack.

Once an error condition has been recovered, the responsibility to handle the error is passed back to the user. If they don't want that they may re-issue the panic by calling `panic(code, 0)`, where the 0 indicates that the same final action (e.g exit) as before the recovery is used.

As examples how this could be used by the C library, there are also augmented versions of storage allocation function, such as defer_malloc. The idea that these will panic if they encounter an out-of-memory condition. That has several effects

• Explicit handling of such error conditions has not to be repeated at every call site.
• Cleanup actions that the user has installed by means of defer clause will be called, e.g to close files or to free large allocations.
• User code may establish a recovery mechanism through recover or alike. This will probably be rarely used by "normal" applications, but security critical applications would get a handle to avoid catastrophes, here.

The important interfaces of this tool are:

• guard prefixes a guarded block
• defer prefixes a defer clause
• break ends a guarded block and executes all its defer clauses
• return unwinds all guarded blocks of the current function and returns to the caller
• exit unwinds all defer clauses of all active function calls of the thread and exits normally
• panic starts global unwinding of all guarded blocks
• recover inside a defer clause stops a panic and provides an error code

The features exit, quick_exit and thrd_exit are overloaded to do the unwinding as soon as this header is included. To be consistent when such code is linked to legacy object files that have been compiled differently, you should wrap the system calls. This can usually be done by providing `-wrap=exit` etc arguments to the linker. (You'd have to check your local manual for that.) This feature may (but shouldn't) be switched off by providing the environment variable DEFER_NO_WRAP to the build.

In a similar way, this implementation also offers to wrap the allocation functions of the C library, but the default here is the other way around: per default malloc and Co are not wrapped. You may enable this by setting DEFER_MALLOC_WRAP. Otherwise, you may use functions defer_malloc and similar to catch allocation errors.

send with panic) are always positive, and that "system" error codes are generally negated errno numbers. To deal with such error codes in a portable way, for all POSIX error codes we provide an equivalent prefixed with DEFER_. E.g there is DEFER_ENOMEM that is the same as ENOMEM on platforms where that exists and some other unused number otherwise.

For signal numbers there is a sophisticated encoding in even smaller negative values. Because the system signal numbers may conflict with errno numbers we cannot use these directly. Instead, for all POSIX signal numbers there is constant that replaces the SIG prefix by a DEFER_ prefix. E.g there is DEFER_HUP to represent the signal SIGHUP regardless if that signal number actually exists on the platform.

Two convenience macros ease the use of this feature, defer_if as a pseudo selection statement, and recover_signal to recover just signals and do nothing if there was none.

C++ has similar features for out-of-order code execution, namely destructors and catch clauses. The defer statement combines those two features into one.

This implementation provides features to translate between these mechanisms such that stack unwinding can be reliably performed in programs that mix C and C++. In particular, if called from C++ code, the C feature itself translates panics and requests for exit at the boundary into C++ exceptions. The "only" thing that C++ applications have to do for this to work is to establish a mark, that effectively the current language is C++. When the C++ code includes <stddefer_codes.h> this is done (by some hackery) for any translation unit that contains the function main.

Nevertheless, to really ensure that resources are released by such a C++ programm it is a good idea to have all exceptions caught. This will ensure that all destructors down to and including in main are called when unwound. An easy way to achieve this is to establish main as a try-catch block:

int main(int argc, char* argv[]) try {
  ... do your stuff ...
} catch (...) {
  throw;
}

The same could be done for functions that are called with threads of their own.

To establish interoperability for C++ code that might be called from C a bit more is needed. A wrapper testing_CPP for a function testing can be established as in this example:

#include "stddefer_codes.h"
#include "stddefer_cpp.h++"

int testing_CPP(int rec) {
  std::defer_boundary boundary; // Must be the first declared variable
  try {
    return testing(rec);
  }
  catch (...) {
    boundary.panic();
  }
  return 0;
}

That is, we need

• a local variable of type std::defer_boundary for which the constructor memorizes the state prior to the call,
• a try/catch clause that guarantees that exceptions are caught and that all the destructors are called when unwound
• a call to the method panic() that either propagates any exception that was caught (if the caller is also C++) or that translates an exception to a panic (if the caller is C).

For the moment there is no automatization of such a wrapper feature.

When used consistently like that, arbitray back and forth translation between C defer and C++ exceptions is performed but has its limits in the expressiveness of the defer error codes. We translate some of the standard codes and exceptions:

Support for other codes might come in the future.

This implementation uses setjmp/longjmp as a main feature to jump to deferred statements within the same function or to unwind the call stack. We distinguishes "jumps" that are known to target the same function (_Defer_shrtjmp) and those that are known to jump to another function on the call stack (_Defer_longjmp). The unwind for a return will usually be all short jumps, whereas exit and other unwinding of the call stack always initiates a long jump.

This implementation is quite hackerish because it uses nested for loops to implement the principal macros defer and guard. It does so to ensure a mostly library only iplementation through macros, that could in principle work with any C compiler. This technique leads to code that is difficult to read and can therefore not be recommended.

Also, we use a intermediate processor for the production of the C (and C++) code called shnell. This uses #pragma annotations for the definition of complicated macros and for "code unrolling". So if you'd want to modify this implementation, you'd have to download shnell and modify the "real" sources.

defer statement

Deferred statements may not themselves contain guarded blocks or other defer clauses, and shall not call functions that may result in a termination of the the execution other than panic. Additionally, such a call to panic must only occur after the current panic state had been tested with recover.

The deferred statement may use any local variable that is visible where the defer is placed and that is still alive when the deferred statement is executed, that is at the end of the surrounding guard or function body. This property is checkable at compile time, and a violation usually results in an abortion of the compilation. This implementation here puts everything in place, such that a deferred statement uses the last-written value for all variables, but the success of that depends on the presence of some synchronization feautures.

If such synchronization features are not available, local variables that may change between the defer itself and the execution of the deferred statement and that are used in the deferred statement must be declared volatile such that the latest value is taken into account. So as a rule of thumb, variables that you use inside a deferred statement should be qualified: const qualified for those where the defer clause should use the original value, and volatile qualified for those that should be used with their latest changes.

The implementation uses allocation as of calloc and free to maintain the list of defer clauses. In case that calloc fails, the defer clause is executed and then the whole execution is unwound by means of panic and an error argument of -DEFER_ENOMEM.

guard compound-statement

If such a block is terminated normally or with a break or continue statement, all deferred statements that have been registered a defer statement are executed in reverse order of their registration. There is also a macro defer_break that can be used in contexts where break or continue statements would refer to an inner loop or switch statement. Also, return, exit, quick_exit and thrd_exit all trigger the execution of deferred statements, up to their respective levels of nestedness of guarded blocks.

Other standard means of non-linear control flow out of or into the block (goto, longjmp, _Exit, abort), do not invoke that mechanism and may result in memory leaks or other damage when used within such a guarded block.

For some of these constructs, there are replacements defer_goto, defer_abort etc that can be used instead.

noreturn void panic(int C);
noreturn void panic(int C, void F(int));

After all deferred statement of the current thread are executed in reverse order in which the defer statements have been encountered, in the last stack frame that has a defer or guard statement, F(C) is executed. Here F is a defer_handler and C is an error code. If omitted, F defaults to an implementation specific defer_handler.

Valid defer handlers are all functions that have the correct signature and that terminate the current thread or the whole execution. Prominent candidates from the C library are exit and thrd_exit.

This unwind chain can be stopped with a recover call within one of the executed deferred statements (or if a defer_if clause is used).

The error code should be negative if this is supposed to map to a system defined error condition as is for errno or a caught signal. Otherwise, all user supplied error numbers should be positive. When unwinding an error condition triggered by the system, the error code will always be negative.

noreturn void defer_exit(int);
noreturn void defer_thrd_exit(int);
noreturn void defer_quick_exit(int);
noreturn void defer__Exit(int);

The functionality of these functions is the same as their un-prefixed C library counterparts, only that instead of terminating the thread or the execution immediately they will unwind the stack of collected deferred statements.

Code that is compiled with the <stddefer.h> header will also just replace calls to the C library functions

noreturn void exit(int);
noreturn void thrd_exit(int);
noreturn void quick_exit(int);

by calls to these wrappers.

Other code that is not compiled with it, may still call the C library functions directly without unwinding. So when you are mixing such units of different origin you should use linker tricks such as -wrap-function=exit to help you replace these calls by the wrapped ones.

int recover(void);

A call to this function must reside within a deferred statement.

This will only stop unwinding of the panic if the error code had not been 0. Normal break, return, thrd_exit, exit etc will not be be stopped and the guarded block, function, thread or program will finish when all the associated deferred statements have been executed.

If the error code is non-zero, execution of the defer clause then continues as if the nearest guarded block had been broken by break. That is, the execution of the defer clauses of that particular guarded block are continued and then execution resumes at the end of that guarded block.

In general, error codes provided by the system will be negative and user provided error codes shall be positive. So to see if a system error occurred you may e.g compare the value to -ERANGE, or, if the error originated from a caught signal to DEFER_SIGNO(SIGTERM).

The provided signal handlers use extensions that might not be available everywhere. In particular, they use the fact that thread-local variables can be accessed from a signal handler.

void* defer_malloc(size_t size);
void* defer_calloc(size_t nmemb, size_t size);
void* defer_realloc(void *ptr, size_t size);
void* defer_aligned_alloc(size_t alignment, size_t size);

The functionality of these functions is the same as their un-prefixed C library counterparts, only that instead of returning a null pointer on failure they will issue a panic. For example, defer_malloc, will execute panic(-DEFER_EINVAL) when called with a size of 0 and execute panic(-DEFER_ENOMEM) when the allocation fails.

This has two advantages. First, when using these functions you will never again forget to check of the result of such a function. Second, subsequent code (if reached) can always assume that the return value is non-null. For example other than for realloc the following is a valid idiom:

void* p = malloc(small);
defer free(p);
... do something with p ...
defer {
    ... save some precious values from p ...
}
p = defer_realloc(p, huge);

The assignment to p is only executed if the call to defer_realloc is successful, and the call to free in the deferred statement will always use the good (= last allocated) value for p. Also, if it fails execution is immediately transferred to the currently deferred statements, and the necessary terminal actions are taken as indicated, first the second deferred statement may save values from p and then p is freed.

• This work is distributed at

https://gustedt.gitlabpages.inria.fr/defer/

• The sources can be found at

https://gitlab.inria.fr/gustedt/defer