There are still some fairly big features we have planned and several breaking changes. This is what I'm thinking for how I'd like things to play out...

These are the biggies we still need for Fan 1.0:

• Symbols: changes how facet and localization work
• DSLs: ability to plugin custom grammars between the <| |> symbols
• String Literals: changes to Str/Regex literals leveraging the DSL work
• AppHome: enhances how Fan environment is installed across multiple directories
• NonNull Field Verification: define how constructors check that non-null fields are set to something
• JavaScript Compiler: needs to reach production maturity
• WebApp: needs to reach production maturity (heavily used in Bespin)
• JavaScript based gfx/fwt: Andy is working on this now to use for Bespin

Hopefully over the course of the next month or two, we'll knock out the key breaking changes. Since SkyFoundry's strategy is dependent on Fan's JS compiler and JS-based FWT, I expect that work will progress rapidly (it is basically Andy's full-time job).

If you think there other things that absolutely must go into 1.0, now is the time to talk about them.

I am hoping that by mid-summer we have a fairly stable language and set of core libraries. After that, I'd like to see a fairly long (at least six month) beta period. During this period we will attempt to keep things stable and avoid adding major new features. Although if we have a great idea that breaks things (such as Symbols), then the door is still open. But I'll be increasingly anal about adding things. Assuming we have a successful beta period and we are happy with the language, then we can declare an official 1.0 release.

After the 1.0 release, we will branch the code and create a 1.1 repo. We will freeze the core 1.0 APIs and only do critical bug fixes for the 1.0 repo. All new development, language enhancements, and APIs will begin in the 1.1 branch.

Thanks for clarifying the schedule. This is exciting!

I'm really excited that so much effort is being directed at making Fan a truly cross-platform environment. The ability to support different platforms with one code base has potential for great cost savings.

So is everyone cool with the proposed feature set for 1.0 then?

If you think there is something else that belongs, now is the time to voice your opinion.

My wish list:

Property change notification, preferably with notification and listener addition/removal baked into the language.

One of the things that bugs me in Java land is the plethora of event listener types and event dispatch protocols. The JavaBeans spec tried to tackle this but it is incomplete and there is no direct language support.

For one, you have to write your own add/remove listener methods. These methods are defined in the spec but are not in any interface so you have to implement them yourself and be meticulous about the signatures:

PropertyChangeSupport changeSupport = new PropertyChangeSupport(this);

public void addPropertyChangeListener(PropertyChangeListener listener) {
  changeSupport.addPropertyChangeListener(listener);
}

public void removePropertyChangeListener(PropertyChangeListener listener) {
  changeSupport.removePropertyChangeListener(listener);
}

and/or

public void addPropertyChangeListener(
      String propertyName, PropertyChangeListener listener) {
  changeSupport.addPropertyChangeListener(listener);
}

public void removePropertyChangeListener(
      String propertyName, PropertyChangeListener listener) {
  changeSupport.removePropertyChangeListener(listener);
}

You also have to fire the property change notification yourself, e.g.:

public void setName(String name) {
  changeSupport.firePropertyChange("name", this.name, this.name = name);
}

The use of strings to identify properties is also brittle and non-refactorable.

In fan, the add / remove listener methods could be standardized around a simple Notify mixin and Event class:

class Event {
  Obj source
  Obj event
}

mixin Notify {
  ** A null event indicates interest in any event
  Void addListener(Obj? event, |Event| listener)
  Void removeListener(Obj? event, |Event| listener)
  Void notifyListeners(Event event)
}

** alternative interface, adopting fwt::EventListeners into sys..

mixin Notify {
  EventListeners onAny(|Event| listener)
  EventListeners onEvent(Obj event, |Event| listener)
  notify(Event event)
}

The Event and Notify would serve as base event classes for all events, including events not based on a field change (like a button click). Field change events would be a specialized class:

class FieldEvent : Event {
  Obj? oldVal
  Obj? newVal
}

Possibly we may want to introduce a FieldNotify extension mixin for type safety, also this is probably redundant:

mixin FieldNotify : Notify {
  Void addFieldListener(Field? field, |FieldEvent| listener)
  Void removeFieldListener(Field? field, |FieldEvent| listener)
}

Finally, we could introduce a new field modifier bound which would wrap the setter in the standard boilerplate:

bound String name

would be desugared to:

String name {
  set {
    notifyListeners( FieldEvent {
      source = this
      event = #name
      oldVal = @name
      newVal = @name = val
    } )
  }
}

.. in the first Notify API, or:

String name {
  set {
    notify( FieldEvent {
      source = this
      event = #name
      oldVal = @name
      newVal = @name = val
    } )
  }
}

.. in the second Notify API.

List / map change notification. This could be readily accomodated with a checked method which wraps the list in a decorator, and which passes all changes to a closure which chooses what to do with that change. This closure may fulfil any number of roles, such as filtering (disallow strings with less than 5 chars) or event notification. A possible API:

class ListChange
{
  Obj? elem
  Int index
  Bool add
  Void applyTo(Obj?[] list)
  {
    .. etc
  }
}

// example usage
Int counter := 0
Obj[] watchedList = Obj[,].checked |wrappedList, change|
{
  counter++
  change.applyTo(wrappedList)
}
echo(counter)         => 0
watchedList.add("a")
echo(counter)         => 1
watchedList.add("b")
echo(counter)         => 2

This is a simple, cheap way of decorating arbitrary lists and maps with specialized behavior. By using the decorator pattern we also avoid having to burden the vanilla list/map implementation with these concerns. The only caveat is that folks who use the checked API will have to understand that direct changes to the vanilla list will not be detected by the decorator, so all changes must be made through the decorator.

Graph-based serialization - the current situation with tree-based serialization is problematic in some situations. I have an idea how to do this using a thread-local whiteboard and a special reference syntax akin to symbol literals. There are a few details I still need to work out before making an official proposal on this one.

A correction to the Notify mixin (alternative version):

mixin Notify {
  EventListeners onAny()
  EventListeners onEvent(Obj event)
  notify(Event event)
}

Thus:

class Thing {
  bound Str name
}

t := Thing { name = "one" }
t.onEvent(Thing#name).add {
  echo("Changed property $it.event from $it.oldVal to $it.newVal")
}
t.name = "two" => "Changed property Thing.name from one to two"

Also it would be a compiler error to declare a bound field in a class without mixing in Notify.

@qualidafial - Thanks for posting those thoughts. This is what I'm thinking:

Field change notification: this is really one piece of a generic data binding framework. Specifically for field notification I've been thinking more along the lines of a facet that works like a Python decorator. But regardless of the actual design, this is something that I eventually want to support. However I don't think it is a critical feature for 1.0. So I think this may or may not happen this year based on how Andy's JavaScript goes.

List/Map notification: still up in the air about this one. I've had in there, taken it out, thought it about some more. I'm increasing thinking that is best done by wrapping the core collections versus having the carry around function references.

Graph Serialization: this is a good thing to talk about, and I'd like to see your proposal. But this one I can a lot of thought to, and it is bigger than just Fan serialization. The abstract Fan model is tree based because it allows the same model to be serialized using Fan's grammar, XML, JSON, YAML, etc. Although this really goes hand-in-hand with a potential data binding framework (so might happen because of that feature).

I'm increasing thinking that is best done by wrapping the core collections

A related thought I've had is that it might be nice to support syntax like Something[] as a generic list of Something as defined by a mixin, not a concrete class. The expressions for new lists and maps would stay the same, but the type specifier would be for the abstract type.

This would allow wrappers while still specifying the type of list or map. I've debated bringing this up, but this seemed like a sufficiently related subject.

The increased abstraction is both a pro and a con. Also, you would need to make sure any method added to List or Map would also (or instead) be added to the generic mixins as concrete methods so as not to break other existing implementations.

Is this interesting enough to break out into a separate thread, or is it a nonstarter?

Is this interesting enough to break out into a separate thread, or is it a nonstarter?

I don't think it is a non-starter per se, although not something I would really want to consider for 1.0. I designed Fan to heavily favor the built-in list and map as the standard way to share collections - that has its pros and cons. It means you can't share a collection such as a linked-list in a standard way, but that is by design - because linked-lists and array backed lists don't share the same performance characteristics. But I can see revisting this issue later - just not something I want to consider for getting a 1.0 release out the door.

However I don't think it is a critical feature for 1.0.

Property change support itself isn't critical, however there are established listener APIs e.g. in fwt that we may want to change to use the Notify mixin. After Fan 1.0 goes gold, this won't be possible without breaking clients and/or having API with two ways to do the same thing.

I'm increasing thinking that is best done by wrapping the core collections versus having the carry around function references.

Exactly--the original list is just a list, the decorator keeps a reference to the function and the original list. All mutator methods called on the decorator get passed to the check function. It is the check function which adds behavior before and/or after passing the change to the original.

This is the way I envision this happening:

class List {
  Void add(V val) {
    // the current implementation
  }

  List checked(|V[],ListChange| check) {
    return CheckedList.make(this,check)
  }
}

class ListChange {
  Obj? val
  Int index
  Bool add
  Void applyTo(Obj?[] list)   {
    .. etc
  }
}

internal class CheckedList : List {
  List target
  |List, ListChange| check

  // Override every mutator to look like this
  Void add(V val) {
    checked(list, ListChange {
      it.val = val
      it.index = size
      it.add = true
    } )
  }
}

Note that ListChange as documented above does not cover all possible changes. It may make sense to introduce a ListDiff class as well to cover move complex changes:

class ListDiff {
  ListChange[] changes
}

Thus a set operation would be broken into two operations: remove the old element and insert the new element. This is what we do in Eclipse data binding and we have helper classes API for walking the changes at a higher semantic level:

class ListDiff {
  Void accept(ListDiffVisitor)
}

mixin ListDiffVisitor {
  abstract Void handleAdd(V val, Int i)

  abstract Void handleRemove(V val, Int i)

  Void handleSet(V oldVal, V newVal, Int i) {
    handleRemove(oldVal, i)
    handleAdd(newVal, i)
  }

  Void handleMove(V val, Int oldI, Int newI) {
    handleRemove(val, oldI)
    handleAdd(val, newI)
  }
}

The abstract Fan model is tree based because it allows the same model to be serialized using Fan's grammar, XML, JSON, YAML, etc.

Using the approach I'm contemplating, the serialization syntax would remain a proper subset of the Fan syntax.

JSON and YAML already support graph based serialization using & markers and * backreferences. XML could be using a reserved attributes e.g. id markers and idref backreferences like Java's XMLEncoder serialization uses.

The basic idea is to introduce a grammar for object markers. These markers would be compiler sugar for undeclared local variables.

Str a := "blah" %%thing%%
Str b := %%thing%%

Note that I'm not in love with the %% syntax but all the pretty options seem to be taken already! :) So please, anybody feel free to suggest something else. The above would desugar to:

Obj? %%thing%% := null
Str a := %%thing%% = "blah"
Str b := thing

When blocks are nested, the variable would be in the scope of the outermost block (i.e. the method block).

a := [1]
a.each { it %%thing%% }

Desugars to:

Obj? %%thing%% := null
a.each { %%thing%% = it }

With this syntax in place, the serialization support could be enhanced to support back-references:

Bank {
  Account {
    number = 1234
    owner = Customer %%Customer1%% {
      name = "John Doe"
    }
  },
  Account {
    number = 4567
    owner = %%Customer1%%
  }
}

Desugars to:

Obj? %%Customer1%% := null
Bank {
  Account {
    number = 1234
    owner = Customer {
      %%Customer1%% = it
      name = "John Doe"
    }
  },
  Account {
    number = 4567
    owner = %%Customer1%%
  }
}

Notice that the %%Customer1%% marker assignment was inlined to the first statement in the it-block. If we assign the marker after executing the it-block, there is a possibility for missing data / stack overflows when the object is serialized. A degenerate example of this is:

class Thing {
  Thing thing
}

Now suppose we want to serialize this:

Thing {
  thing = it
}

OutStream.writeObj will output this for a while:

Thing { thing = Thing { thing = Thing { ...

Until you run out of storage space or get a stack overflow. Another problem is that if you desugar this:

Thing %%Thing1%% {
  thing = %%Thing1%%
}

Into this:

Obj? %%Thing1%% := null
%%Thing1%% = Thing {
  thing = %%Thing1%%
}

Then %%Thing1%% hasn't been initialized when the thing = %%Thing1%% assignment happens, so null is assigned instead.

By inlining the assignment at the beginning the it-block I think we avoid all these problems. Thus the above example:

Thing %%Thing1%% {
  thing = %%Thing1%%
}

desugars to:

Obj? %%Thing1%% := null
Thing {
  %%Thing1%% = it
  thing = %%Thing1%%
}

What do you guys think?

Edit:

JSON and YAML already support graph based serialization using & markers and * backreferences.

I looked again and YAML does support object references but JSON does not. However per the JSON article on Wikipedia there are possibly a few conventions to choose from.

Property change support itself isn't critical, however there are established listener APIs e.g. in fwt that we may want to change to use the Notify mixin.

Good point, I'll have to give that some thought.

Regarding the graph based serialization - I'd probably think about the more along the lines of facets and that syntax. Although at this point, I don't really want to dive into design proposals - I just want to have a conversation about what we are leaving out and how that might impact future versions. In this case, I don't see that we can't tackle this in the future without impacting what we have now.

I'll go ahead and split out my proposals into their own topics so we can discuss them separately.

Out of those three issues, the only concern left that in my mind should be tackled for 1.0 is to have a unified notification scheme e.g. via the Notify mixin.

I'm going to glance over the docs to see if there are any downstream clients for notifications other than fwt.

Are Regex patterns platform-independent yet? As painful as it seems, I think this would be important for 1.0.

@tompalmer - we still use the native platform implementations directly. We'll have to consider Javascript as well I think if we decide to make this truly platform-neutral (which I think is important as well).

A thought on Regexes. Everyone is used to their own platform's dialects. Maybe we should allow for a dialect configuration at the object or pod level so everyone can use what they know, but write the bridge code to let it run on all platforms. Then, as a platform optimization, the .NET regexes would be native on .NET and the Java regexes would be native on JVM, but each would be seamlessly emulated on the other.

We'll have to consider Javascript as well

Good point.

A thought on Regexes. Everyone is used to their own platform's dialects. Maybe we should allow for a dialect configuration at the object or pod level

Hmm. I see why it would be nice to support both options. I'd rather see platform-neutral by default, personally, and I'd rather not use magic means for the configuration. Just my own thoughts.

Anyway, I think there are other threads on this, so I won't go deep here. Just glad to see some possibility of getting this compatibility in for 1.0.