This is the fourth of a series of articles on what I’ve learned about Erlang (and Elixir) from writing Erl2ex, an Erlang-to-Elixir transpiler. This week we take a look at variable scoping, comparing Elixir’s caret operator with Erlang’s single assignment, and we’ll explore a few surprises lurking in how variables are scoped in the presence of different language constructs.
Is Elixir data really immutable?
Immutability is one of the most important features of many functional languages, including Erlang and Elixir. Preventing changes to existing data simplifies concurrency management, enables a variety of performance optimizations, and aligns your code with well-understood architectural patterns.
But wait… one thing we may have learned early on was that, while Erlang enforces single-assignment—any given variable name can be bound only once—Elixir allows variables to be reassigned.
Does this mean that Elixir actually does allow mutation? After all, we’re modifying the value of variable “a” above, right?
As we look at variable binding and scoping, we should start by emphasizing an important distinction. Elixir data is just as immutable as Erlang data. In the example, the values
:bar remain unchanged, immutable, in memory. If you are working with one of those values, it cannot be changed out from under you. All that is changing is which value the variable “a” points to.
In this article, we’ll see that variable binding and scoping behavior in Elixir and Erlang are sometimes different, and can occasionally be confusing. However, it is important to remember that both languages retain a commitment to data immutability that underlies much of their shared concurrency and reliability stories.
Most modern languages employ some form of lexical scoping. This means that the hierarchy in a program’s structure dictates the scope of a variable. A variable defined in a block remains visible within that block, and typically within sub-blocks, but cannot be accessed outside the block (unless it is exported somehow). Here is an example in Elixir:
If a variable in an inner scope has the same name as one in an outer scope, the inner variable hides the outer one; the inner is said to “shadow” the outer. To see this more clearly, let’s begin with a Ruby method that sums the values in an array. (Yes, you could also use Enumerable.inject, but suppose you wrote it in a more “imperative” style.)
This code relies on the fact that Ruby allows you to modify the “partial_sum” variable from inside the “do” block, a common pattern in more “imperative” languages. Trying to translate this “directly” into Elixir would not work:
Of course, you would be correct to criticize such Elixir code as not properly employing a functional style. Idiomatic Elixir would likely use
Enum.reduce(). And of course, we’d expect that attempting such code in Erlang would fail completely, because you can’t rebind the variable at all. And that’s the point. Variable scoping and binding behavior are key distinctions in different languages, and often force you to structure your code differently. As we’ll see, the implications can be subtle and surprising.
The caret or the stack
In January, Jose Valim, creator of Elixir, wrote an article on immutability and variable rebinding in Elixir and Erlang. Jose asked the question, is one language more “safe” than the other due to variable rebinding? He shows that, while immutability makes both languages are “safe” in terms of shielding data against clobbering by other processes, both languages are actually vulnerable to bugs that can be introduced if someone changes variable bindings by modifying the code. Elixir variables bind by default, so a value can change unexpectedly if you change the code to insert a new binding. Erlang variables either bind or reference depending on the current state, so behavior of an expression can change unexpectedly if you insert a new binding.
Jose’s article may feel a bit arcane on first glance, but it is well worth understanding because it highlights the variable life cycle and scope that will let us explore some idiosyncracies of both languages.
When an Erlang variable appears on the left side of a match, the behavior depends on the context: if this is the first appearance of the variable in that scope, it is bound at that time, otherwise it is merely referenced. Elixir, on the other hand, forces you to be explicit in choosing whether to bind a variable or reference its earlier value. A variable on the left side of a match is bound, unless it has a caret, in which case it is referenced.
Here’s a simple example in Erlang:
The two matches in the function do different things: the first binds the variable A, but the second only references the variable (throwing a match error if the value of A is different from the value of Y). If we were to translate this to Elixir, it might look like this. Notice how the semantic difference is now explicit.
This becomes ever more important as we study the behavior in the presence of nested scopes. For example, as we saw earlier, an Elixir function definition creates an inner scope that shadows outer variables. Is the same true in Erlang? Sort of. Consider this example:
The “A” variable in function “F” is in an inner scope that no longer exists at the end of “foo”, so we’d expect that A can then be bound in the outer scope, and the value 2 would be returned. This is indeed the case.
The Elixir equivalent of the above function is:
However, what if we swapped the order of the two bindings of variable “A”?
Now it throws a match error. Erlang now considers that “inner” A variable to be a reference to the outer A, and it fails because
1 != 2. Perhaps that was unexpected at first glance, but it kind-of makes sense. Previously, when the outer “A” variable was bound second, the scopes of the two variables never overlap. However, now when the outer “A” variable is bound first, it is lexically still in scope inside function “F”. Erlang has to resolve this conflict somehow, and it could have done so in two ways. The inner variable could reference the outer variable. Or it could shadow the outer variable and create a new local variable inside function “F”. Erlang’s strategy is to choose the former.
Elixir, on the other hand, allows you, the programmer, to make the choice by including or omitting the caret. The Elixir equivalent of the above Erlang code is:
However, remember the choice that Erlang had: either refrence the outer variable or create a local variable that shadows the outer variable. You have the same choice in Elixir with the caret operator. If you omit the caret in Elixir, you are not only deciding to bind rather than reference, but you are actually changing the scope of that inner variable. It does not rebind or affect the outer “a” variable, but it creates a local “a” inside the inner function.
So far so good? Well now we get to the surprising part…
Exposing the exports
Scope creation, especially when interacting with control structures, tends to be messy in pretty much any language. Erlang and Elixir are, unfortunately, no exception. We saw above that function scopes can partially “leak” in certain cases. Here’s another example.
Remember from above that an Erlang variable in an “inner” function remains confined to its inner scope as long as that variable wasn’t previously bound.
This is not true for some other control structures.
In the above case, the “A” variables in the case statement are “exported” to their surrounding scope. Variable “A” has already been bound at the time of the later match, so that match will succeed only if you pass 1 to the function. In other words, its Elixir equivalent is:
Elixir does force you to make it explicit that a match is taking place, so I think it is a bit more clear what is going on than in Erlang. However, the fact remains that variables inside the case statment are leaking out into the surrounding scope in both languages.
Similarly, if you bind the variable A before the case statement, then A cannot be bound inside the case statement, because the outer variable is still in scope:
The equivalent in Elixir:
One more quirk to explore. Recall that if you bind a variable before defining an inner function, that variable is visible within the function, and Erlang cannot bind it again.
However, if the “inner” variable is bound as an argument, Erlang treats it as a new variable, and shadows the outer one.
Again, Elixir, through its pin operator, gives you the choice of either behavior: referencing the outer variable or creating a new shadowing variable in the inner scope. Here is the Elixir equivalent of the above:
If you include the caret, you reference the outer variable:
If you’re thoroughly confused at this point, I don’t blame you. Because of behavior like this, it is generally good practice, in both languages, to avoid reusing variable names, even if they live in different scopes and you expect one to shadow the other. The rules are subtle and can be surprising, and for the sake of clarity of your code, it’s best to avoid corner cases like these that we’ve been exploring.
Where to go from here
Chasing down all the special cases around variable scopes was an enormous headache for Erl2ex. It’s important to be aware that there are some surprises, in both languages.
If you haven’t read Jose’s article, I recommend it. It’s a useful read.
The good news is that Elixir is trying to simplify its scoping rules by deprecating the confusing constructs, especially exporting variables from control structures such as “if” and “case” statements. Elixir 1.2 already emits a warning at compile time for some cases, and Elixir 1.3 will deprecate additional cases. Daniel Perez recently blogged about some of the upcoming changes.
Next time, we’ll look at the exception handling in the two languages and how they relate. Feel free to browse the index of articles in this series, and stay tuned for more on Erlang and Elixir’s family ties.