Overview

This is a proposal to change the syntax of curly brace blocks: closures and with-blocks. This is something we have talked about for many months, but I have never found a solution that felt good enough to actually implement until now (barring any issues raised which I haven't considered).

Closures vs With-Blocks

Today the syntax |,| {...} is required to write a no parameter closure. We use the {...} syntax to indicate with-blocks. The problem with this choice of syntax is that it prevents us from using {...} to write closures which look like "built-in control structures":

// eh, not so hot
atomic |,| { ..... }

// looking good
atomic { .... }

There are tons of previous discussions on this topic.

Let me propose a simple alternative:

{ .... }         // closure
expr : { .... }  // with-block

The change is to require a colon after an expression to indicate that a curly brace block is a with-block:

// the FWT example using the new syntax
Window :
{
  size = Size(300,200)
  Label : { text = "Hello world"; halign=Halign.center }
}.open

Unlike using a dot such as expr.{}, I think colon expr:{} works no matter how you do your brace styling.

Serialization

Along with this change, we will also change the serialization syntax used by InStream.readObj and OutStream.writeObj to use colons. Serialization and with-blocks are designed to be consistent.

Non-Local Returns

The problem we've had with {...} closures was that it wasn't obvious what return meant. I think we can solve this well with a couple changes:

1. Change everywhere return is used inside a closure to break
2. Allow closures to use return as a non-local return (using exceptions similar to Java closure proposals)

This simplifies things to mean return always exits the method and break always exits the block (be it for, while, or closure).

Expr Based Returns

Today most closures are fairly simple and only contain a single return statement at the end of the block (soon to be break). If we allow an expression to be used as an implied break inside a closure, then the use of break to mean return from closure isn't so bad:

// today
files.sort |File a, File b->Int| { return a.modified <=> b.modified }

// using break
files.sort |File a, File b->Int| { break a.modified <=> b.modified }

// using expr only
files.sort |File a, File b->Int| { a.modified <=> b.modified }

It is this change coupled with the change to break that makes the whole thing palatable for me.

Recap

So to recap my proposal:

1. replace with-block syntax from expr {...} to expr : {...}
2. replace return with break inside all closures
3. do a build to allow everyone to update their code
4. drop the |,| from no parameter closures
5. implement non-local returns from closures
6. allow expressions to be used as break from closure

The primary reason driving this change to get to a syntax where {...} can be used for a no parameter closure. This in turns enables alternate DSLs and control structures where |,| is syntactic noise.

My 2 cents..

I think with.{...} is more intuitive than with:{...} since colon means labels for me. And spelled out it would be a dot (which is why it is more logical/intuitive).

I'm not too fond of break as short return from a closure. Couldn't return mean the same thing as in Java and another keyword be used for "long return"? I bet you the difference between short and long returns will be a source of many many bugs so expressiveness should be more important than short syntax here IMO. How would you, using break, "return short" from within a for loop for instance?

I think if the closure is a single statement (rval) no return should be needed, otherwise it should be mandatory. That goes for methods as well btw.

Cheers, Mikael Grev

These all look like excellent changes to me. I rather like the new with-block syntax because the colon suggests "with" to me; e.g. "here's the stuff I want you to do with this object". It also makes it clearer to the reader that a with block is being used, which is always a good thing.

After these changes are made, and statements are expressions, we're just two more steps away from being able to implement loops as ordinary functions:

1. A lazy keyword that wraps a parameter as a thunk at the caller site, exactly as it's implemented in the D programming language (to get the value of a lazy parameter, one must use parentheses, e.g. foo()).
2. Changing the for loop separators from semicolons to commas.

Void for(lazy Obj initial, lazy Obj condition, lazy Obj action) {
    initial()

    if (condition()) {
        action()

        for(null, condition, action)
    }
}

for (i := 0, i < 100, i++) {
    ...
}

Not that this is how you would actually implement the constructs (this implementation would lead to stack overflow without tail call optimization), but you know a language is powerful when you can implement such basic constructs in the language itself.

Couldn't return mean the same thing as in Java and another keyword be used for "long return"?

The beauty of this proposal is that it makes the language consistent: break does what you think it does, whether in a loop or in a closure.

Of course, it's possible to take the language in inconsistent directions, but why? To help Java programmers who are use to anonymous classes? In the long term, the language will attract more developers with a simpler, consistent syntax.

I bet you the difference between short and long returns will be a source of many many bugs so expressiveness should be more important than short syntax here IMO.

Your suggestion forces a developer to memorize more rules. It's simpler to remember, "break gets me out of the code block," and "return gets me out of the method". Yes, you have to memorize those two rules, but they don't have exceptions. Sometimes a break with convention is what you need to create a more usable language. :-)

I think with.{...} is more intuitive than with:{...} since colon means labels for me.

Tom or John originally suggested the expr.{}, but I'd say it is out because it doesn't work if you like Allman styling bracing (which we do - and lets stay away from that subject :-)

Frame:
{
  // works well
}

Frame.
{
  // doesn't work so hot
}

A lazy keyword that wraps a parameter as a thunk at the caller site, exactly as it's implemented in the D programming language

Having that would also let you implement && and || with short circuiting as normal method calls (instead of the special case they are today).

+1

I think it resolves quite a bit of previous discussion and moves Fan in a more elegant direction.

I'm on board with all this.

It's simpler to remember, "break gets me out of the code block," and "return gets me out of the method".

Sure, but a closure, for the non-language purists (99%), looks more like a function/method than a normal for/while/do/switch code block. The compiler treats it like a code block but developers are very unknowing about stuff like this (in general).

Sure, but a closure, for the non-language purists (99%), looks more like a function/method than a normal for/while/do/switch code block.

Just so we are clear, the whole point of this proposal is to make closures look more like for/while/switch. This allows you to use closures to invent your own control blocks:

atomic 
{
  if (account1.balance < amount) break
  account1.subtract(amount)
  account2.add(amount)
}

No matter how you cut it, embedding functions inside a method has some confusing semantics. But I believe that the principle of closure blocks acting like other looping blocks is actually the most consistent model. I should be able to change a while loop to an each closure and get the same behavior with regard to return and break.

Sure, but a closure, for the non-language purists (99%), looks more like a function/method than a normal for/while/do/switch code block.

Actually, when a developer sees a closure, and has no prior experience with them, he or she generally assumes that a return is non-local. (Example forthcoming.)

As I recall, we've even had a few people post to this site who knew that Fan's closures didn't have non-local returns, but were still confused because the closure syntax is so minimal that it's easy to overlook (as indeed it should be, that's the whole point of closures!).

The compiler treats it like a code block but developers are very unknowing about stuff like this (in general).

I think the result of this change will be less confusion, not more. And for people new to the whole closure thing, who are coming to Fan from a language without them, it will be a lot less confusion, fewer rules, and greater consistency.

Having that would also let you implement && and || with short circuiting as normal method calls (instead of the special case they are today).

In fact, the only basic construct you couldn't implement with the language (as far as I can see) is if statements. Because of that pesky dangling else. :-)

OK, you guys have convinced me. I'm on board.

Cheers, Mikael

Closure does look better to me for the first time :-)

I have to say that I don't really like it. It's certainly alien to the bulk of developers coding today. I actually suspect that this decision may be the key to where Fan goes and how successful it is.

BTW, I am comfortable with single expression closures and colon with-blocks.

The problem is not that the solution above isn't elegant or powerful - it is. The problem is that most developers don't understand or appreciate that elegance. Proof? Well look at the popularity of languages that follow the proposed style vs those that don't. Is Fan really the language to succeed at popularizing this style when Smalltalk and functional programming et al didn't?

Javascript and C++ (next version) are following the model of short-returns with parameter-based closures. I believe C# does too, and lots of others just allow a single expression and so act like short-returns. I'd strongly argue that Fan needs to stay on the successful and popular side of this fence!

Why are short-returns natural? I think its because as a developer your intent is to write a what amounts to a method (anonymous, and low ceremony). The concept of passing around blocks of code is not natural.

So, what about long-returns within a parameter passed closure? Its a very rare requirement.

This proposal also seems a little broken conceptually:

list.each |Str str->Void| {
  str = doSomething(str);
  if (str == null) {
    break;
  }
}

The break doesn't do what you think it should. In the proposal (IIUC) the break will simply end this iteration of the loop and then cause the next iteration to occur. But that really isn't what anyone from any mainstream language with break in would expect. It looks like a loop, so I'd expect break to end the loop.

Further, the rule as to what break links to is undefined:

Void myMethod() {
  count := 0
  doSomething();
  foreach |Str str->Void| {
    try {
      if (...) {
        break;
      }
      count++
    } catch (Err err) {
      // ignore
    }
  }
}

So, what does break connect to? Why does it connect to the foreach and not the if or the try? The rule is to find the nearest closure or genuine for/while? Thats hard.

Overall, I think that the DSL concept tackles the need for control structures. I don't think we need to support control structures using closures at all in Fan.

DSLs get compiled in place and thus we avoid long-returns and their horrible exception based implementation altogether. So a DSL that is actually a thin wrapper around Fan code can be written for using a resource or the actor model or for-each.

And, the higher barrier of entry prevents most developers from coding control statements (and preventing this is a key factor in many coding shops).

Overall, I think that following Javascript, C#, C++00x etc is the way to go here:

list.each |Str str| #{
  if (a) {
    return "A"
  } else {
    return "B"
  }
}

list.each function(Str str) {
  if (a) {
    return "A"
  } else {
    return "B"
  }
}

list.each fn(Str str) {
  if (a) {
    return "A"
  } else {
    return "B"
  }
}

You can certainly implement this change in two steps too - where the first step is allowing single expression closures without using return.

Valid points Stephen, I'll have to go back and sleep on this one.

I actually suspect that this decision may be the key to where Fan goes and how successful it is.

I don't think this particular issue is anywhere close to a deal-breaker, regardless of the outcome. Most closures are tiny and used as callbacks or glue. I'd be very surprised if less than 90% of the usage of closures simply omit any return-style keyword.

Overall, I think that the DSL concept tackles the need for control structures.

I actually agree that the DSL concept is far more powerful and completely subsumes the majority of my use cases for what we were calling unified with blocks. But keep in mind that an average developer will not be creating DSL plug-ins (though I imagine the average developer will be using them). They're simply too complex to write. Yet, regular developers seem to have little trouble using Ruby to create custom constructs. That's why there's still a market for something between textbook Fan and the all-powerful DSL plug-ins.

The break doesn't do what you think it should.

I agree this is a problem -- in fact, I thought of this very use case yesterday, but didn't have time to write about it. break should break from the each loop in the same way that break breaks from a for loop. In fact, such functionality is required to implement breaking for loops, contra my little toy example posted earlier.

So, what does break connect to? Why does it connect to the foreach and not the if or the try?

I agree, to be consistent you would have to connect to the if, which would make break pretty useless. break is useful as a way to get out of an iteration. I'd really like it to function the same way in each or any other similar construct.

I want these to work the same way:

[0, 1, 2, 3, 4].each |Int v| {
    if (v > 3) break;
}

for (v := 0; v < 5; v++) {
    if (v > 3) break;
}

Moreover, if statements ever become expressions, I'd want these to work the same way, too:

val1 := [0, 1, 2, 3, 4].each |Int v| {
    if (v > 3) break 3;
}

val2 := for (v := 0; v < 5; v++) {
    if (v > 3) break 3;
}

// val1 == val2 == 3

Honestly, the only way I can see the above happening is if break actually throws some kind of BreakException, which can be caught or not depending on the needs of the function invoking the closure. I don't see a way of implementing the above using return values, because you want to distinguish between normal exit of a closure and breaking exit, and the one special return value is null, which may be returned by the closure anyway.

Now if break throws an exception, then you need a way to return a value in the normal fashion, because for many closures (those not used in an iteration), breaking just doesn't make sense. That brings us back to return.

I'm interested in seeing what others have to say about this, but based on my strong desire for the above two constructs to behave identically, I'm less inclined to want break to act as a normal return in a closure.

Stephen,

Thanks for taking the time to go into detail. You said much better what I tried to say.

Unfortunately I am afraid that the "language purists" will hijack a very promising language. Basically all "pure" languages end up in a nice somewhere.

Please don't go Scala with Fan.

Cheers, Mikael

My main objection to break as return was its inherent "unnaturalness". They are all just functions, so return has very clear and well understood semantics. Brian managed to convince me otherwise yesterday, though his argument escapes me at the moment.

And I will second Mikael here - we've been very careful to keep Fan evolutionary over revolutionary far the most part, so we should tread carefully here.

Unfortunately I am afraid that the "language purists" will hijack a very promising language.

The point is not to hijack the language, but to make control flow more obvious and intuitive to developers. This proposal is motivated by the very minimal closure syntax Fan has, especially for no-argument, no-return closures; and by the fact that you can specify the closure following the method call, as in Ruby.

Take a look at the following:

Void with(File file, |,| closure) {
   closure.call()

   file.close
}

Bool readFile() {
   ...

   with(file) |,| {
      if (file.read() != VERSION_2_3) return false;

      ...
   }

   return true;
}

The above code looks like it should work, but the |,| actually introduces a closure, so the return is local (and therefore discarded, if it's permitted at all). The situation is much better when the closure takes arguments and/or returns a different value from the enclosing function.

JavaScript in particular doesn't have this problem because of the bulky (but quite suggestive) keyword function, which introduces every anonymous function; and because if you pass an anonymous function to another function, the anonymous function is completely embedded in the parameter list, and enclosed in the parentheses required to invoke the function. These are clear visual signals that aren't present in Fan.

Now that said, I'm not sure this proposal is the right solution for this "problem", because it introduces inconsistencies elsewhere. It's good to see lots of feedback, in any case.

I actually agree that the DSL concept is far more powerful and completely subsumes the majority of my use cases for what we were calling unified with blocks.

Not sure I agree with this. If the only thing inside the DSL is a block of Fan code, then by definition that is a closure. There are some really common cases for this, two of them being C# style using statement which closes resources for you and verifyErr which tests that a specific exception is thrown. So I'm not sure we say DSLs are some panacea for all these cases - in fact since under the covers they would be implemented with closures anyhow we would still have to solve the problem.

Regardless I really hate the |,| {...} syntax and want to get rid of it. But there are some good points raised here regarding break versus return etc.

So let's separate that issue. If we take that out of the equation do we all agree that:

1. with-blocks should be changed to expr : {...}
2. the last statement of a closure is implied to be a return
3. closures without parameters should be changed to simply {...}

Is there any disagreement on these issues?

I think where the return issue (syntax and non-local returns) comes into play is concerning the fact that |,| {...} was more obvious that a closure was being declared than {...}.

I agree with Stephen that non-local returns (return from method instead of closure) isn't very common - in fact I think all my use cases have been solved with eachWhile.

So another approach is that instead of trying to make the exit syntax of methods and closures consistent, we make them deliberately inconsistent. This makes it crystal clear what is happening even when using extremely light syntax such as {...}. The best way to do that is completely disallow return to be used inside a closure and use another keyword. Then if you are confused and try to return from a closure it is a compiler error. I have no idea what that keyword might be, some suggestions: ret, leave, yield, go. I kind of like yield, but a bit inconsistent with how other languages use it as a generator (or maybe not inconsistent?)

What do you guys think about that?

If the only thing inside the DSL is a block of Fan code, then by definition that is a closure.

I'd disagree. A closure implies that the user written Fan code is compiled as a class and passed to the DSL handling code. Instead, I see the user-written code as being compiled directly in place:

// source code
TimeForPerformance("MyCategory") <|
  doSomethingNeedingTiming();
|>

// de-sugared source code embedded in original class file
start := DateTime.now();
doSomethingNeedingTiming();
end := DateTime.now();
log("MyCategory", end - start);

No need for long-returns or any complications.

There are some really common cases for this, two of them being C# style using statement which closes resources for you and verifyErr which tests that a specific exception is thrown.

Then maybe the syntax for DSLs needs to be cleaner for Fan code:

Using (foo) {
  ...
}

This is still a DSL, but because it only contains Fan code it can use curlies. (And this might imply something about constructor-with-blocks...)

do we all agree that: > 1. with-blocks should be changed to expr : {...} 2. the last statement of a closure is implied to be a return 3. closures without parameters should be changed to simply {...}

I have no great problem with (1), although I fear I may be missing something.

For (2), I only really support single expression closures without return.

For (3), I believe that if we have both control statement closures and parameter based closures then you need two syntaxes. In FCM closures I use the # symbol:

list.each |Str str| #{
  doStuff();
  // return foo - would return to nearest #
}

(return is defined to return to nearest method, where # is a symbol for an anonymous method)

Control statement closures are then allowed, as they wouldn't use the #:

using (foo) {
  // return - would return to nearest method, which might be anonymous
}

Thus, two different brackets for two different use cases.

Note that you could swap the symbols around, so the control statement variation gets the symbol and the parameter one doesn't. However, I think that it is conceptually very useful to have a visual hook that the return binds to when scan reading the code.

I suggested other alternatives earlier, BTW - fn or function (but not lambda...)

non-local returns (return from method instead of closure) isn't very common

If you want it, I've suggested throw return, as that is what actually happens, and makes the potential runtime error less surprising.

The best way to do that is completely disallow return to be used inside a closure and use another keyword. Then if you are confused and try to return from a closure it is a compiler error.

I don't think this is necessary if you provide the visual hook I'm talking of. Javascript and C# don't have any problems with nested returns, why should we?

Here is my basic philosophy: if the code within a "DSL" is strictly Fan code, then it is not a DSL. Personally I find that an abuse of the feature, indicating a problem in the core language - namely that closures (the built-in way to a capture a block of code) aren't powerful enough. DSLs should be used to write non-Fan code. And remember that the code within DSLs will by and large be ignored by IDEs. Obviously people can do what they want with DSLs, but that is shaping up to my view of when you should and should not use a DSL.

For (3), I believe that if we have both control statement closures and parameter based closures then you need two syntaxes.

This isn't really a path I want to go down. There should only be one type of general purpose closure.

However, I think that it is conceptually very useful to have a visual hook

I believe this is the core issue. How badly do people want the simple {...} syntax? Personally I have a strong desire to move that direction, and I don't believe any special decorator is need - it only adds visual noise.

And I believe by changing return inside a closure to another keyword such as yield or ret would actually be the best solution (let's leave non-local returns out of it). Or we could just change to the use the {} closure syntax and leave return completely alone. This is how Groovy does it I believe.

The |,| actually doesn't really bother me, and I don't run across it that often anyways. If anything I would prefer simply || over leaving it off.

The more I think about it, I really don't like changing the semantics of return in closures. A closure is a method, and I don't see a good reason to create these special rules for it. Plus how would I pass you a method instead of a true closure, where one uses return and the other uses break?

I have not yet seen a compelling reason to change the current design.

Not sure I agree with this. If the only thing inside the DSL is a block of Fan code, then by definition that is a closure.

I agree. I'm referring strictly to my main use cases: STM, XML generation, message matching, parsing, etc., all of which are nicely solved by DSLs. As I said in my reply to Stephen, I don't think DSLs replace the need for flexible syntax. DSLs are giant wrecking balls, but sometimes a sledgehammer will work just fine (and in any case, Joe average is not going to be able to create a wrecking ball of his own).

with-blocks should be changed to expr : {...}

Yes. This is much clearer, and makes it look like : is a method accepting a no-arg closure -- a unifying, if not altogether true, concept (and with more flexible binding, it could actually be true).

the last statement of a closure is implied to be a return

Yes, 100% agreed here. Fan has a declarative coding style already, and adding implicit returns takes it further in this direction.

closures without parameters should be changed to simply {...}

I agree. Right now, |,| is noise and is as likely to be missed when scanning the code as Stephen's # {. It also impedes the creation of language-like constructs.

DSLs should be used to write non-Fan code.

Exactly. It's possible to misuse DSLs for adding features to Fan's syntax, but that's clearly abuse. Small snippets of Fan embedded in a DSL (such as SQL query or STM) is fine, but pure Fan code with a little syntax sugar is not fine.

And I believe by changing return inside a closure to another keyword such as yield or ret would actually be the best solution (let's leave non-local returns out of it).

Agreed here as well. If we use some keyword other than return, then no developers coming from other languages will be confused -- the first time they try to return a value from a closure code block, they'll get a compiler error.

Moreover, with this approach, there's no need to scan for a 1 or 3 character marker at the beginning of the code block, since if you care about where the return value is going, you just look for the return statements and see what keyword is being used.

Now I don't particularly like the ret keyword because it's just a shortened version of return. It would be like using return2. I prefer a completely different keyword, to indicate a completely different semantic.

The keyword yield is actually very descriptive, but the problem is that you might want to use yield for generators. And when used for generators, yield can appear anywhere, even in a normal method -- its scope is not limited to closures -- and when it appears in a normal method it has the exact same effect as return (albeit with a hidden stack save that is restored for the next iteration).

Other options: give, render, giveback, offer, provide, returnup, giveup, yieldup (up being the direction the return value is going).

A closure is a method, and I don't see a good reason to create these special rules for it.

In Fan, a closure is a function that looks nothing like a method. In JavaScript, there are just functions, and they all have the exact same syntax (though sometimes you leave out a function name), so using a single keyword makes sense.

Fan has already made the choice to give methods and closures different syntax, and has gone further by making the closure syntax more minimal, almost non-existent in the case of a no-arg, no-return closure. So in Fan, you face different problems than you do in JavaScript.

I've already given examples of code that looks like it should be doing one thing, but is doing something else entirely. That's a serious problem in my book.

John, sounds like you are on the same page as me:

• change with-block to be expr : {...}
• change no arg closures to be simply {...}
• pick new keyword to use for return inside closure

I am really liking that design. The only question is what keyword? I really like yield best, but as you point out is this a poor choice given it is used in generators.

In languages like Ruby, yield is used to pass a value back to a block. But in Fan we pass in a closure and you call the function explicitly. So would we ever use yield in that context?

Things like give, render, giveback, offer, provide don't work well for void closures which are exits. Returnup is ok, but I don't really love it.

If we don't use yield, then I'd say my second choice is leave. It has the correct connotations of both exiting the closure and potentially returning a value. Other choices: exit, halt, depart, cede.

Or just we could just stick with return like Groovy does.

If we don't use yield, then I'd say my second choice is leave. It has the correct connotations of both exiting the closure and potentially returning a value. Other choices: exit, halt, depart, cede.

I really like leave. As soon as I saw that, I wondered why I didn't think of it. You're leaving the current block but not returning from the method, and it works well enough for void methods, too.

Or just we could just stick with return like Groovy does.

-1 to that. return makes sense in JavaScript. I don't think it makes sense in Groovy or Fan.

So many sides of the coin. Hmm. Here are features from my ideal language:

• Closures and ordinary blocks look the same. That includes allowing multi-clause blocks (if/elseif/else). That some blocks are inlined should be implementation details.
• Parameters for blocks look a lot like parameters for top-level methods.
• The return keyword is not needed for returning values. It always returns from top-level or otherwise clearly labeled methods, not anonymous blocks.
• Any break or continue feature would work the same for all blocks.
• I use labeled break and continue a lot, and I'd be tempted to require labels for any use of these features.
• John's "lazy" feature would be vital, though I might represent it differently.

From a Fan perspective, I don't know what's best. It could stay as is, or the proposed changes could be made. Just a few thoughts:

• If something looks like a block (i.e., no |,|), it should act like a block.
• If break exits closures, it needs to exit if when curly braces are used or somehow be clear why it doesn't.
• I'm not sure I like a separate returnish keyword for closures.
• Glad to hear everyone agrees at least with single-expression closure auto-return.

The approach with long-returns and an alternate keyword is really very different from the one in use today in Java, C#, Javascript, Groovy... And I believe its different for a reason.

Closures interacting with Built-in statements

Lets examine leave. Its defined as "leave the nearest closure".

Void doStuff() {
  list.each |Str str| {
    while (process(str) {
      if (str == null) {
        leave
      }
      str = handle(str)
    }
  }
}

So, leave will "leave" the each closure as expected. But why doesn't it "leave" the while "closure"? Or why doesn't it "leave" the if "closure"?

The answer is that as soon as your language defines any kind of built-in control construct the "simple" rule breaks. Absolutely and completely. (And no-arg closures in Groovy are an example of this broken approach).

There are thus two directions to fix the break:

1. have no built-in control structures (Smalltalk, Lisp, other functional languages???)
2. accept that the developer has to be able to work out which are library implemented (closures) and which are built-in control statements (Javascript, C#, Java, ...)

Fan is an evolutionary language. It has built-in control statements. Thus, I claim that developers must accept responsibility for understanding what is a closure and what isn't - direction #2. (And equally I claim that those who find this uncomfortable come from the <1% of developers who understand FP)

Once that is understood and accepted, it becomes obvious that a developer needs to be able to find a closure when reading the code. There must be a visual hook.

I don't particularly care whether it is |,|, #, function, fn or something else, but you need to be able to find the closure.

And in case you weren't following, none of that was dependent on whether the return keyword is return or ret or leave. I simply claim that once you've understood the requirement for the visual hook, return becomes much more natural (and that is why its used in all the other big languages).

BTW, the same fundamental problem applies to break and continue wrt library functions. The developer has to learn the library to know which of the code blocks captures the break/'continue'. And that isn't pretty. Hence I claim that break and continue shouldn't be used with closures.

Control statements

So, what about implementing control statements in libraries? Well, I'm simply arguing that feature should be evaluated separately.

Why? Well, there is a requirement for library control statements wrt return/'break'/'continue' that is different to the requirement within closures. (ie. for control statements, the meaning of these is absolutely well defined)

The first question is whether it is needed at all. Fan has survived without it so far, as has Java. A using built-in statement would address much of the requirement. Beyond that we have DSLs. However, I believe that there could be room for a middle ground - it is just not closures in the sense described above (unification isn't viable in a language with built-in statements).

Personally, I would consider something like a macro, perhaps linked into the code similarly to a DSL. The key to my strategy would be two keywords, which make these library-written control statements into visible built-in statements:

do Atomic {
  if (account1.balance >= amount) {
    account1.subtract(amount)
    account2.add(amount)
  }
  // return works as per normal if/try/catch statement
}

for FileEachLine(file) {
  doStuff()
  // break/continue/return work as per a normal for/while statement
}

These are not closures. The code is not passed to a method, it is compiled in place and surrounded by code emitted by the controlling rule (Atomic/FileEachLine). This avoids any long-return issues and their nasty implementation hack via exceptions.

Summary

I understand the desire to reduce language features and to unify. But I think it should follow the maxim of as simple as possible and no simpler. I hope I've explained more clearly why having no-arg closures be {...} is a very serious problem (and reproduces the flaw of Groovy).

I also hope that I've given some indication as to why this matters and how crucial it is to Fan. One direction leads to a clone of Scala, the other to an evolutionary derivative of Javascript, Java and C#. I know which I want to code in.

+1 Stephen.

The answer is that as soon as your language defines any kind of built-in control construct the "simple" rule breaks. Absolutely and completely.

That's not necessarily true.

Suppose, for example, that when calling a closure, you can catch BreakException, LeaveException, ContinueException, and ReturnException. If you don't catch them, the compiler will execute the default behavior.

In this case, you can imagine that while intercepts BreakException and ContinueException, but passes through the others. In other words, each construct gets to decide how to handle the break / continue / return / leave behavior of the closure that it invokes.

This is just one example of a way to move toward consistency and self-hosting. Who's to say there aren't others?

I don't particularly care whether it is |,|, #, function, fn or something else, but you need to be able to find the closure.

Personally, I'd rather move in the direction of there being no distinction between built-in control structures and user control structures.

+1 Stephen - said much better than I would have.

Closures are functions, yes they have a different scope and different signature syntax, but still a function. And they should act that way IMO.

I really like Stephen's idea of tackling control structures from another angle - since thats really what we're trying to - not mask a function as a block - but actually create a block.

Fan is an evolutionary language. It has built-in control statements. Thus, I claim that developers must accept responsibility for understanding what is a closure and what isn't - direction #2.

I am not all that interested in moving towards some Smalltalk like purity where all control structures are implemented in the language itself. So I believe you identify a key point here Stephen. The simple fact of the matter is that curly braces are overloaded to have several meanings (including in all of the built-in control statements). Let look at what we have today:

Statement       Allowed Exits
---------       -------------
Void f() {...}  return from method
|,| {...}       return from closure
if {...}        return
for() {...}     return, break, continue
while() {...}   return, break, continue
expr {...}      no control flow inside with-block

Rules according to my proposal:

Statement       Allowed Exits
---------       -------------
Void f() {...}  return from method
{...}           leave from closure
if {...}        return/leave
for() {...}     return/leave, break, continue
while() {...}   return/leave, break, continue
expr : {...}    no control flow inside with-block

The key issue here is what does a given set of curly braces mean? I can see there is an argument that having |,| {...} is a bit easier to anchor a given set of curly braces. But at the same time (as someone who has written a ton of Fan code), the |,| becomes very noisy.

At the same time, I've never been super happy with using return inside a closure because even I find it a bit confusing. Moving to return from method and leave from closure makes everything really crystal clear.

Hence I claim that break and continue shouldn't be used with closures.

I agree. My proposal has nothing to do with break, continue, or non-local returns from closures. My proposal is simply to omit the |,| syntax on no-arg closures and switch from using return within a closure to use the leave keyword.

Personally, I would consider something like a macro, perhaps linked into the code similarly to a DSL.

Interesting idea, and certainly something we could look at in the future - but I think it is a bit orthogonal to this discussion because the easiest way to compose blocks of code is using closures (a feature we already have). While the idea might seem elegant, an implementation not using closures would be really complicated.

I like JohnDG's idea, but I doubt that continue / ContinueException can be implemented efficiently.

(JohnDG) In other words, each construct gets to decide how to handle the break / continue / return / leave behavior of the closure that it invokes.

And thats fine from the perspective of the implementer of the library. But, it ignores the perspective of the user of the API. The problem is how the user knows, just by reading the code in front of them and without consulting library documentation, which methods capture break / continue / return / leave and which don't. This is the key failing of closure-based library driven control flow.

(Brian) The simple fact of the matter is that curly braces are overloaded to have several meanings (including in all of the built-in control statements).

True. The question is not whether there are different rules, but how easy they are to learn, and crucially how easy the resulting code is to read.

At the same time, I've never been super happy with using return inside a closure because even I find it a bit confusing. Moving to return from method and leave from closure makes everything really crystal clear.

And along comes a developer from Javascript, C#, Java, Groovy..., and they will use return. And it won't do what they expect. And that is very confusing. (Because while you might make return a compiler error initially, eventually with your approach it would become a long-return).

The irony is that 95%+ of your worries with return in a closure being "a bit confusing" will go away when you allow single expression closures to be an implied result.

Lets consider a loop:

for (i := 0; i < list.size; i++) {
  if (i == 6) leave
}
list.each |Int i| {
  if (i == 6) leave
}

These are not equivalent. The leave in for would leave the loop (a la break). The leave in list.each will start the next iteration of the loop (a la continue). Yuck.

> something like a macro Interesting idea, ...[but] the easiest way to compose blocks of code is using closures (a feature we already have).

My contention is that coding something like atomic, using or withfile is actually modelling an extension to the core set of built-in statements. And it should be modelled as such. It isn't a block of code as used by closures. There are different requirements at work, and it should be modelled independently rather than forced together.

Finally, lets consider a combined example.

Void process() {
  str := ""
  while (handle(str)) {
    list.each |Str item| {
      if (comparisonService(item, str)) {
        leave
      }
      matched(str)
    }
    str = nextString()
  }
}

So, the leave will exit back to the closure and continue the next loop. Now lets introduce a timer to check for performance:

Void process() {
  str := ""
  while (handle(str)) {
    list.each |Str item| {
      time {
        if (comparisonService(item, str)) {
          leave
        }
      }
      matched(str)
    }
    str = nextString()
  }
}

This has broken the program. The matched(str) method now gets called all the time, irrespective of the result from the comparisonService.

With my proposed approach, we model time as a built-in construct (because that is the intended usage) and we model each as a closure (because that is the intended usage). Basic

Void process() {
  str := ""
  while (handle(str)) {
    list.each #(Str item) {
      do time {
        if (comparisonService(item, str)) {
          return
        }
      }
      matched(str)
    }
    str = nextString()
  }
}

And thats fine from the perspective of the implementer of the library. But, it ignores the perspective of the user of the API. The problem is how the user knows, just by reading the code in front of them and without consulting library documentation, which methods capture break / continue / return / leave and which don't.

Convention is one solution: break exits loop-like constructs, continue continues loop-like constructs, leave passes through loop-like constructs, etc. When designing a library, you should always be thinking how to conform to the user's expectations of what the code will do.

My contention is that coding something like atomic, using or withfile is actually modeling an extension to the core set of built-in statements.

It's actually impossible to implement atomic using a macro. It requires a compiler plug-in. And viewing closures as a chunk of code, it seems to me pretty obvious to use a closure when implementing using or withfile: you want to surround the execution of a hunk of code with boilerplate. That's textbook classic example of how lambda functions can be used to extract duplication.

+1 for Stephen Please don't go the Scala way... It really feels like you want to show off the strength of the language by removing 3 characters that have a meaning, a reason and helps everyone: The reader, The IDE, the API user and the API writer.

My main complaints are:

1. I'm currently writing an API in Fan and I for the moment the closures types it accepts are |,|. But for how long? Now with |,| (or whatever |->Void| type info telling me and my user it's a closure), and can do the refactoring using IDE. But with {...} I'm stuck.
2. I prefer static typing because I can express myself and communicate with others a lot more efficiently with nice Types. For me good typing (especially with a good IDE), is the best friend of all programmers. |,| is my type, it's true it's a stupid type, but I like it.
3. Why forcing an API writer to manage, break, yield, leave, return, when he can provide a clean eachWhile with good typing. The API readability/expectation is related to the complexity (power) of the "closure contract".
4. The power of closure is the Scope transparency, not the control structure. I know language purist will hate me, but all the problems above are cleanly solve with a local variable which is for 10 times more expressive than break a <=> b

Conclusion:

• expr : {...} with no control flow inside with-block: Good for me
• No long-returns
• Keep |,|

Hope you'll keep Fan readable and use the power in your hands wisely.

I think this issue has been hashed out pretty well along several dimensions including many of the previous discussions around trying unifying with-blocks and closures. Andy and I discussed it during a brainstorming session on Thur and he is quite opposed to introducing another keyword.

I think my conclusion is to leave things exactly like there are with the exception of allowing return to be omitted in some situations.

We will continue to require the |,| syntax for no argument closures. I don't really like it because it is syntax noise. But at the same time I can't deny it is consistent syntax noise which clearly introduces a closure.

Since we are going to keep the |,| for no arg closures, I think it simplest to keep the return keyword versus introducing a new keyword. Non-local breaks and returns will not be supported. I find this hardly ever to be an issue if you choose the right list or map method to use. Although that is the sort of thing fairly easy to introduce in the future without breaking backward compatibility.

With-blocks will continue to use the current expr {...} syntax. I use with-blocks extensively, and I don't see a good reason to introduce an additional char since we don't need to now. This is especially nice regard to the serialization syntax, since the current syntax is really clean for human reading/writing.

I do still think we should allow an expression based return. I think a nice simple rule is that the return value of a method or a closure can be a single expression as long as it is the only statement in the block. This means that return is required if you have more than one statement. We can also relax the rules later, but this seems a simple, safe way to start.

I don't see a good reason to introduce an additional char since we don't need to now.

Because then : becomes a method. You don't have to do anything with that now -- but it might be useful in the future. Also, in a developer's mind, an operator : as a method is an easier mental model than a chunk of code that follows an expression (which puzzled me greatly the first time I saw it in Fan code).

We can also relax the rules later, but this seems a simple, safe way to start.

Sure, the most common use case is one-line expression-based closures, anyway.

Since we are going to keep the |,| for no arg closures, I think it simplest to keep the return keyword versus introducing a new keyword.

That's definitely simplest, though I would suggest that using leave is equivalent to buying an option, which lets you decide the meaning of return at a later date. If you decide that return always means local return, you can't easily go back. But if you use leave, then later, you can decide that return means either local return or non-local return, and it won't impact the code (even if you decided that return meant local return, and deprecated/removed leave, developers could update their code with a simple search & replace operation).

You don't have to do anything with that now -- but it might be useful in the future.

I agree that it does allow a bit of future proofing, but at the expense of syntax noise. If we are going to keep the |,|, then I'd prefer to keep with-blocks as clean as possible.

That's definitely simplest, though I would suggest that using leave is equivalent to buying an option

Now that we are going to keep the |,| I don't particularly care one or the other. I would be happy leaving it as return or changing it to leave. Given my ambivalence, I'll leave the decision to Andy (last time I talked to him was solidly for keeping it as return).