Teaching old assert() new Tricks

Many languages have assert(). This post describes how assert() is implemented in Next Generation Shell and how these aspects/features are consistent with the rest of the language.

Implied in the above is that the ideas below might not be a good fit for assert() in your favorite language or framework.

Note that assert(EXPR, PATTERN) in NGS is same as EXPR.assert(PATTERN). It’s called UFCS.

Matching to a Pattern

Following the observation that in many cases predicate callbacks just compare the given value to a pattern, most if not all methods in NGS that were previously taking a predicate argument were upgraded to take a pattern argument instead. Following that development, assert(EXPR) got the assert(EXPR, PATTERN) flavor, which turned out to be used way more frequently than the “regular” assert(EXPR).

Usage of patterns in assert() is completely aligned with the rest of the language, where many other methods support pattern arguments.

Example 1:

# Originally
children.assert(Repeat(Element), "All children of Element must be of type Element. Have ${children}")

# For purposes of this post but also works
children.assert(Repeat(Element))

Example 2:

assert(log_opt, AnyOf(true, Str), InvalidArgument('log: option must be either true or a string'))

Evaluate to the Original Value

assert(EXPR, PATTERN), can of course throw an exception. When it doesn’t, it evaluates to EXPR, implementing fluent interface. This leads to the following idiomatic usages:

  • In-line checking: something.assert(...).method1(...).assert(...).method2(...)
  • Assertion before assignment: myvar = something.method1(...).method2().assert(...)

In object oriented languages, in order to chain assert()s, you would have to have the .assert() as a method in each class (could be inherited maybe). That would be weird to have such “pollution”. In NGS, since there are no classes, the assert() method is defined outside of any class, like all other methods, and works with any types of EXPR and PATTERN that implement pattern matching (EXPR =~ PATTERN).

In NGS, whenever there is a good candidate to be returned from a method, it is returned. The fluent interface of assert() is no exception to this rule and is aligned with the rest of the language in this aspect too.

Example 1 (in-line):

# Excerpt
c_execve(..., options.get('env', ENV).assert(Hash, 'Environment variables must be a Hash').Strs())

Example 2 (just before assignment):

# Inside retry()'s "body" callback
ret = curl_gh_api(wf_run.url).assert({'conclusion': AnyOf(null, 'success')}, 'Expected successful build but it failed')
ret.conclusion == 'success'  # Reporting for retry()

Speaking about exceptions …

Checking for Exceptions

The need arose for testing for exceptions that were expected to be thrown. The solution in NGS was to implement assert() that would work with callable EXPR and a subtype of the Exception type. It calls EXPR, catches any exceptions and checks whether it’s the expected type of exception.

assert() taking different types of arguments follows filling-the-matrix principle, which says that any method should support as many types of arguments as reasonable (a programmer could potentially guess the use or at least easily remember once learned).

Example 1:

{ decode_json("1xx") }.assert(JsonDecodeFail)

Example 2:

F guard_no() { guard 0 }

guard_no.assert(MethodNotFound)

The following paragraph is completely optional. It only explains the guard above. Skipping it would not impede understanding of the rest of this post.

In the last example, guard is used. NGS relies heavily on multiple dispatch. A particular flavor is employed where the search for matching method is a bottom-up scan with invocation whenever all the arguments matched the parameters. If your method is more specific and would like to report “it’s not me, please continue the scan” it should use guard EXPR. If EXPR evaluates to true, execution inside the method continues. If EXPR evaluates to false, the scan continues (upwards). If no methods are left to scan, MethodNotFound exception is thrown.

Domain Specific

Following once again filling-the-matrix principle, assert(EXPR) flavor can also take some domain specific types.

  • assert(File("my_config")) – throws if the file is not present
  • assert(Program("dig")) – throws if the program is not installed

When EXPR is converted to boolean inside assert(), the Bool method is called.

Defining Bool(f:File) to check that the file is present and defining Bool(p:Program) to check whether the program is installed makes the assertions above possible.

Filling-the-matrix principle was followed here again.

Additionally, this section shows that NGS is “a general purpose programming language with domain-specific facilities” – the best phrase I have to describe NGS.

Example 1:

WEB_SERVER_DIR = Dir(".." / "ngs-web-ui").assert()

Example 2:

assert(Program('jq'), "encode_json() failed to do 'pretty' formatting - jq program is missing")

The Retry Counterpart

NGS was primarily designed for “DevOps”y tasks. As such retry() fits right into the standard library. One of the retry()‘s arguments is a callback – named body – that optionally “does the work” and must always check whether we are done. body callback returns a boolean:

  • true for success and therefore retry() exits
  • false for failure and therefore retry() continues to the next attempt (or fails if number of retries was exceeded).

One of the thoughts was to use assert()s to report failure from body. Using assert()s in this situation would allow ergonomic skipping of remaining steps in the callback, potentially. Practically, using assert(EXPR, PATTERN)provides better error reporting as both EXPR and PATTERN appear in the exception.

This idea was not aligned with orthogonality principle though: retry should deal with retrying, not exceptions handling. retry() could catch unrelated exceptions inadvertently when using assert() in the callback.

Meet retry_assert() which throws exceptions specifically caught by retry(). If it’s not the last attempt by retry(), nothing happens, and next attempt is started (after delay specified in one of retry()‘s parameters). On the other hand, if it is the last attempt by retry(), an exception is thrown similarly to one produced by assert().

retry() with retry_assert() demonstrate “domain specific facilities” here once again.

retry_assert() is aligned with orthogonality principle which NGS tries to follow.

Example 1:

# In retry()'s "body" callback

get_session(session_id).retry_assert({'status': 'RUNNING'})

Example 2:

# In retry()'s "body" callback

get_session(_session.id).assert({'status': IfExists(AnyOf('RUNNING', 'PENDING'))}).retry_assert({'status': 'RUNNING'}, 'Session did not resume')

Note the mix of assert() and retry_assert()in the last example. Use assert() if you detected a situation that would not resolve with more attempts. Use retry_assert() to convey that further attempts should be tried, unless the maximum is reached, of course.


Hope it was an interesting read. Let me know.

Leave a comment