This is not a proposal per se, just some random thoughts. They aren't complete, but I think I am onto an idea that really might work well...

Basically facets are meta-data on pods, types, and slots. In effect they are another "dimension" to the namespace. But today facets are strictly a static dimension, a single facet name/value pair for a given pod, type, or slot.

But sometimes it might be helpful to have a "meta-data dimension" on a per instance basis. The classic case is observable properties which must store both a value and the list of listeners. Ideally what we want to do is add another namespace dimension under slots. For the sake of consistency I will use the term facets for this dimension.

Sometimes facets are static and sometimes they are instance based. For example:

class Foo
{
  @transient
  @listeners
  Str someField
}

In this case transient can effectively be static, someField is always transient regardless of instances. However, I desire a listeners instance per instance of Foo. That could be defined in the facet itself:

static Bool transient
Listeners listeners

But that isn't ideal because you can easily imagine that some facets might be static or instance based on their context such as unit which might always be fixed for a given field, or might be configurable on a per instance basis. In fact, in previous systems I've had use cases where transient would be an instance facet so that serialization of a given field can be customized on a per instance basis.

So perhaps we use two different symbols - # for static facets and @ instance facets:

#transient
@listeners
Str someField

That might work, but we still need a way to access these facets in code. How should we model that? Suppose that facets aren't meta-data attached to the slot, but are actually slots on a subclass of Field or Method for that particular slot? So the above code compiles into:

class Foo$someField : Field
{
  static const Bool transient
  Listeners listeners
}

Now I use only slot reflection:

// old style
Foo#.field("someField").facet(@transient)

// transient is now just a slot on customized Field
Foo#.field("someField").field("transient").get
Foo#.field("someField")->transient->get

That is pretty nice because perhaps the whole issue with symbols maybe goes away. But now we definitely need the notion of "bound" fields as suggested by Stephen in #631:

Type#     =>  unbound Type
obj#      =>  bound Type (not sure what this means)
Type#slot =>  unbound Slot
obj#slot  =>  bound Slot
#slot     =>  in static convenience for EnclosingType#slot (unbound)
              in instance convenience for this#slot (bound)

So now I can access facets in my code just using a slot literal and the dot operator (with compile time safety):

obj#someField.listeners.add |Event e| { echo(e) }

This syntax can also be used to solve the field storage problem by an implicit val facet on all Fields which models the storage location:

Str foo { set { #foo.val = val } }

I like this approach because it brings meta-programming back around to using normal OO classes. In effect facets just become fields on a custom Type, Field, or Method subclass.

Assume for a second that facets can trap on their method or field access (which is required to implement field listeners). Then we can solve the configuration and localization problems by just trapping a static method call:

#Localized
static Str title()

#Config(10sec)
static Duration timeout()

Maybe the Config class looks something like this:

class Config : Facet
{
  Obj? trapMethod(Method m) { Repo.readSymbolsCached(m.symbolUri) }
} 

I definitely like the idea of taking config, localization, etc out of the core symbols and making it a normal OO class to be used by anybody.

Problems:

• What are facets? They are instances of a Type now eh?
• Is static and instance best done with symbol or actual keyword?
• How to private, protected figure in? You would like to say val is just a private facet
• What would a bound Type mean? Any use cases?
• In order to pull off Field subclass above, Field couldn't be const?

This is definitely a good direction. I'll reply more when I have time.

Yes, I think this is on the way. I'd suggest however that the destination is more towards what I call properties. (I've been using properties in Java via code generation for 9+ years, and have monitored the various open source projects that attempt to tackle the issues. Fan has a chance here to get it right, and dramatically change the game.)

For Java, and most other OO languages, the OO is effectively incomplete. It stops at the class and doesn't continue to the fields (and methods) of the class. By this, I mean that it isn't possible in standard Java to obtain an object that represents the field and its current value as an object. The term I use for this is a "property".

Where I think your suggestion above goes wrong is in defining your new concept as a subclass of Field. It isn't

Field is still a very useful concept as a singleton that holds the structural information about the code. This gets used in reflection. It also gets used in some aspects of binding. (When binding a table where each row is an object, you want to bind the Field from the object, not a "per instance field".)

Thus, there are two concepts - Field and Property (or BoundField, but "property" is more common). Conceptually, this is as follows:

// source code
class Person {
  Str surname
}
// generated code
class Person {
  Person$Surname surname
}
class Person$Surname {
  Field field := Person#surname
  Person parent
  Str value
}

Note that the field is still Field, and is unchanged from now (as it is useful as described above, and needs to stay as a const). Note also that the property holds a reference to the field rather than being subclassed from it.

"Annotations"/"Facets" then become conceptualy easy. They simply store any state in the property object:

// source code
class Person {
  @Listeners
  @ErrorCodeHolder
  Str? surname
}
// generated code
class Person {
  Person$Surname surname
}
class Person$Surname : Listeners, ErrorCodeHolder {
  Field field := Person#surname
  Person parent
  Str? value
}
mixin ErrorCodeHolder {
  Str? errorCode    // pretend this generates a field in the implementing class
}

While there are multiple possible implementations, I've suggested a mixin based one above. So, each annotation/facet is conceptually/actually a mixin. This seems to fit conceptually with Fan - an annotation/facet is simply a mixin to the property.

So, what about "static facets". Well, these would be conceptually a mixin with no state:

mixin Transient {
  Boolean transient() { true }
}

Next, we have the syntax issue to tackle to re-simplify the common case of just accessing the value. If the above were implemented naively, then accessing the value of the field becomes difficult:

person := Person()
person.surname.value = "Colebourne"
echo(person.surname.value)

So, we have to have syntax that allows access to the three elements - the value, the property and the field.

person.surname    // accesses the value
person#surname    // accesses the property
Person#surname    // accesses the field

(Perhaps the property access should use a different symbol, but this is a minor point).

Now, we can access everything we want:

person.surname = "Colebourne"
person#surname.errorCode = "ERROR_TOO_LONG"
bindTableColumn(Person#surname)

Config/Localization should be able to represented more as an API at that point

@Config
static Duration timeout := 10sec

(Here the Config mixin would get run when the class is loaded for the first time, and it has the opportunity to read its config file and change the static value)

The final issue is performance. A naive implementation as detailed above will have performance implementations. Each time an instance of Person is created, the JVM has to create an instance of each of the associated property objects. I've done this in a system I worked on, and this had really lousy performance (in creation and GC).

Thus, a better physical implementation is to have all the state on the main object and only create the property instance on demand:

// source code
class Person {
  @ErrorCodeHolder
  Str? surname
}
// generated code
class Person {
  Str? surname
  Str? surname$errorCode
  Person$Surname surname$property() {return Person$Surname(this)}
}
class Person$Surname : ErrorCodeHolder {
  Field field := Person#surname
  Person parent;
  Person$Surname(Person parent) {this.parent = parent}
  Str? value {
    get { parent.surname }
    set { parent.surname = val }
  }
  Str? errorCode {
    get { parent.surname$errorCode }
    set { parent.surname$errorCode = val }
  }
}

Although it seems like a lot of temporary property objects might get created, this is actually preferred by the garbage collector (small, short-lived objects with constant state), and it saves a heap of memory.

Oh, and one more thing. The same basic idea applies to methods too. (I'd call the linked object an "operation", instead of a "property"). Again, facets would effectively be mixins.

Questions:

• does the current concept of Symbols exist at all? (not sure what they are for right now)
• can facet mixins have facets on their fields/methods? (simpler to say no)
• how do facets apply to a type using the above technique? (this could be the so-called "bound type", although there are issues with that)
• how do facets apply to a pod using the above technique?

In general, I'm sort of liking this idea of facets as fields on some specific type. Great ideas here.

It might be worth having a way to name the faceted field/method type, too. That way you could keep things statically typed when passing them around, but I'm not sure the best way to do this.

Static vs. instance facets might be better distinguished by the keyword static for clarity, but I'm not sure how to make it look good.

One option might be to achieve this might be to use ordinary named type for field/method types and apply these instead of individual facets piecemeal:

class Whatever: Transient {
  static const Int something := 5
  readonly Listeners listeners
  private someOtherField or method ...
}

class SomeClass {
  @Whatever
  Str fieldWithSpecialWhateverAttached := ""
}

But maybe the flexibility of piecemeal is nicer, and this doesn't quite seem to do the job. Just considering an option.

I also like the current lowercase facets in Fan better than the uppercase annotations in Java. They don't compete as much with the main definition.

Stephen brings up a good point, are these fields or some new "property" abstraction? I have designed 3 previous systems that made them separate "property" abstractions (and likewise with methods as actions/operations). For example functions of properties that I've used before:

• listeners/data binding
• security checks
• audit logging
• validation

But I've felt this was more because the underlying language required multiple concepts to pull all this together. One of the things I wanted to do with Fan is let any field support this sort of functionality without having to move to some new abstraction. For example, it doesn't make sense to me to require a "property" abstraction to mark a field as transient. So I really want to try to model fields as fields and method as methods - I want to avoid adding new abstractions.

The mixin approach is really nice - I had not made it that far in my thinking but it would really pull the meta-data model towards all the OO features. For example, now I could check if a field is transient with just the is operator:

field.facet("transient") == true  // old way
field is Transient                // new way

That feels really right to me, not to mention it provides all sorts of new ways to leverage the existing type system and how HotSpot will optimize.

I'm kind of thinking that the general problem statement is "how do I create a custom Field or Method subclass for each of my slots?" Given that problem statement why not really allow a full type declaration along with each field or method? Obviously we want to keep the simple case simple, but I'm thinking we should embrace the full set of lang features and lang consistency here. But I have no idea what the syntax might actually look like.

@Brian, @Stephen, excellent and clear discussion. I don't know exactly yet what it is, this idea you're building, but I want it.

Somewhere in here is a correct (by which I mean "useful") set of abstractions.

As it stands now, slots are object-local references to other objects -- doing away with value-types makes that true for fields, and considering a method to conceptually be a Func makes it true for methods.

Assigning a value (really another object) to a field is really just copying its handle. If I get Brian's idea, assignment (= operator) becomes a call to set on the object's field:

class MyClass {
  Int x   ==>   Field{type=Int#; name="x"}
  x = y   ==>   this.slot("x").set(y)
}

so you'd get listener or bound behavior by overriding set in MyField, and somehow indicating that the Int x slot is a MyField.

I don't know what the syntax for that would look like; but jodastephens mixin idea could maybe be applied to a slot the same way it's applied to a class:

mixin Listener
{
  abstract Obj val {
    virtual get { ... }     // automatically applied when  
    virtual set { ... }     // concrete impl. of val is assigned?
  }
}
class MyClass : Obj, MyMixin 
{
  Int, Listener x     // x is Int, x is Listener, #x is Field{type=Int#}
  x = y               // x.val.set(y)  
}

Huh. I keep talking, gonna start getting confused. I'll shut up now.

I think we're on to something here. I'll need to digest it a bit, but the overall symbols design never felt right to me, while this seems to "click" alot better.

If you can implement this simply, that would be beautiful !

OK, here is a strawman on potential syntax based upon the general problem "how can I make my pod, type, field, or methods first class type definitions?".

One key issue: do these definitions make sense before the main type, method, or field definition or after? Convention started in C# with Attributes was before the primary definition, and imitated by Java Annotation and Fan Facets. But as this meta-data turns into a full type definition, I think it makes a lot of sense to come after the primary definition. But I'd like to hear other opinions since I could see the flip side too.

If you assume after, then ideally we'd like some bit of syntax that introduced this definition after a type/method body or field definition. A keyword seems to make sense to mimic the class keyword for class definitions. So maybe four new keywords podclass, typeclass, fieldclass, and methodclass (or potentially one defclass keyword). This keyword is followed a list of extension types and then an optional body.

For example to declare a transient field:

Str name
fieldclass Transient

To declare a transient field which extends the listener field:

Str name
fieldclass LisenerField, Transient

With a body definition:

Str name
fieldclass ListenerField
{
  override Void onSet() { echo("name is now $name") }
}

To add decorator functions to a method:

Void someMethod() 
{ 
  ... 
}
methodclass 
{
  override Void onCall() { echo("$qname called!") }
}

Adding meta-data for types:

class Foo
{
  ...
}
typeclass Serializable, Collection

Adding meta-data for pods (there is only the podclass definition):

podclass somePod 
{
  // still not sure how to declare static meta-data which can
  // be relfected before instantiating the class, maybe with const field:
  override const Depend[] depends := [Depend("sys 1.0")]
}

Facets and symbols would be removed completely and replaced with only the pod/type/slot OO model. Being able to remove those non-core abstractions and focus all the language's power on the core pod/type/slot model seems like a really powerful idea.

I don't think there is a good way to "couple" that stuff, but think it makes more sense to include it inside the defs:

class Foo
{
  typeclass Serializable, Collection

  Str name
  {
    fieldclass ListenerField 
    {
      override Void onSet() { echo("name is now $name") }
    }
  }
}

That seems to group things better IMO. This is how I was originally thinking about this as well from the angle of anonymous inner classes (which aren't quite the same thing).

+1 on the keyword vs. symbol - not sure xxxclass is the best term, but don't have a better name yet.

I don't like the names as typeclass could be easily confused with type classes.

And I'm with andy on putting the definitions inside.

I like Brian's proposal for syntax.

Here's my train of thought. There's already the field get and set definitions that provide some precedent for blocks after fields (from the docs here):

Int id := 0
{
  get { echo("get id"); return *id }
  set { echo("set id"); *id = val }
}

I presume those get and set methods ought to look more like standard methods under this proposal, for consistency:

Int id := 0
{
  Int get() { echo("get id"); return *id }
  Void set(Int val) { echo("set id"); *id = val }
}

This kind of block (similar to Andy's block proposal since a block directly follows the definition) looks great for fields, but it wouldn't be as clear on methods since they already have a trailing block. Also, it would be nice to have the methods directly inside the block and have get/set peers to other methods. And something semi-common ought to work for both fields and methods. I think that leads to something like Brian's direction.

So I instead would expect the new get/set to look like this even:

Int id := 0
fieldclass
{
  Int get() { echo("get id"); return *id }
  Void set(Int val) { echo("set id"); *id = val }
}

And I think that inline could be nice syntax for simple cases:

Str name fieldclass Transient

I also like the new look for applying Field and Method classes defined elsewhere (as normal classes or mixins). Here's a pretended generalization of the example from the docs (formatted like above for easier comparison):

class EchoField: StorageField
{
  Val get() { echo("get $name"); return super.get }
  Void set(Val val) { echo("set $name"); super.set(val) }
}

I pretended StorageField is a Field class that provides storage. Note that I'm using Val as a generic type parameter here, though I haven't defined it with generics. Field might ought to become a generic type if the new concept we're discussing here goes through. The generic type itself would be automatically applied by the type of the field in question? Seems like there's some bending of rules necessary to make all this work.

Added in a later edit -> Here's the example of applying EchoField to the original id field example to complete the transformation:

Int id := 0 fieldclass EchoField

And I also agree with Stephen that it could be nice to cheat on the implementation and let the data sit on the containing class to avoid having lots of extra long-lived objects. I'm not 100% sure what's the right answer for efficiency, but I think issues like that should be considered. Some profiling for different kinds of usage patterns could also be helpful.

+1 keyword, I'm starting to think symbols are fairly well loaded by now.

Andy's is good, the fieldclass block is co-equal with the get{} and set{} blocks now.

What about methods? I guess the similar idea would be to treat the normal method body we have now as being an unnamed methodbody block...

class Foo
{
  typeclass { ... }

  Str name { get {} set {} fieldclass{} } 

  Void rename()
  {
    methodclass { override Void onExecute() {...} }
    name = "foo"
  }
}

Here instead of adding an extra level of indentation and making the normal method body be an unnamed block, the methodclass block gets moved inside the method body.

How about making Brian's new keywords be similar to the super keyword in a constructor def? So:

class Foo : Obj, MyMixin; typeclass Serializable, Collection
{
  Str name :
    fieldclass ListenerField { override Void onSet() {...} }
    get { /*normal field accessor*/ }
    set { @name = val }

  Void rename() :
    methodclass ListenerMethod { override Void onExecute() {...} }
  {
    name = "foo"
  }
}   

Just trying to get my head around the potential syntax. Don't know what I like yet.

Andy's is good, the fieldclass block is co-equal with the get{} and set{} blocks now.

Why not have get and set be methods inside the fieldclass? That seems to unify the whole concept super well.

Well, I dislike all the xxxclass keywords (horrible) and the suffix location for this info. Contrast the two next to one another:

// suffix
class Foo {
  ...
}
typeclass
  Serializable,
  Collection,
  Connections([DbConnection("Test", "Test"), EmailConnection("SMTP")])

// prefix
@Serializable
@Collection
@Connections([DbConnection("Test", "Test"), EmailConnection("SMTP")])
class Foo {
  ...
}

As can be seen, its the additional data that gets passed in that makes it a lot more complex. And with the first choice, all that useful (vital!) info will be off the bottom of the IDE page.

I understand where the idea comes from. With fields, there already is a postfix for get/set overrides.

But do we need that feature anymore?

Why not make all get/set overrides full classes written in a general way:

// suffix and inline
Str name
fieldclass ListenerField {
  override Void onSet() { echo("name is now $name") }
}

// prefix and separate
@EchoSet
Str name

mixin EchoSet : Property {
  override Void onSet() { echo("$field.name is now $val") }
}

Yes, its more work. But its also more reusable. And I don't believe that the manual writing of get/set actually is a common task at all! By making it a little harder, we discourage bad practices, yet make it easier to do the right thing (the generic solution). Plus the overall language looses a feature, making it simpler again.

Do we really have use cases that require inline definitions?

Also, I do think we need to keep some distinction between meta-data and the core of the class/field. They are used for different purposes and with different aims. Otherwise, why do classes not implement Serializable or Collection as mixins today?

So, my proposal remains pretty much as is. Users only write regular classes/mixins/fields. They annotate classes/fields with other mixins (which might need some tweaks). There are no manual get/set override blocks. And behind the scenes, the compiler stitches together the parts necessary.

Finally, I can't work out what the desire is to keep these as subclasses of Field. Consider the field name - are you proposing that it is a static on the field (as there would be multiple instances rather than a singleton)?

All the "property" concept is, is to say that the "static" part is named a "field" and the instance part is named a "property", achieved by holding a reference to the "field" in the "property". This is much more flexible as it allows users to hold and use the "field" part as an object.

Just realized also that if get and set do become slots in the defclass (as I'm proposing so far), then that effectively creates diamond inheritance issues (super-fields vs. inherited defclass get and set implementations). Happily, Fan already has ways of dealing with that.

In any case, are there rules about defclass assignments for overridden slots or subtypes? Do the defclass definitions need to be compatible? Are they inherited automatically?

By the way, I think Kevin's placement of the defclass could be nice. I think they should be kept short, so earlier on (but indented) could keep them visible but unintrusive.

How would you apply static metadata, by the way? Something like this?

class MyClass
  defclass {static const Str author := "Tom"}
{
  ...
}

Seems a bit worder than today:

@author="Tom"
class MyClass
{
  ...
}

Also, would static fields conflict with static definitions in defclasses for supertypes/slots? That is, could I say this (especially if defclasses are inherited)?

class AnotherClass: MyClass
  defclass {static const Str author := "Someone Else"}
{
  ...
}

My previous comment was written before I read Stephen's comment, by the way. Just saying for clarity.

All the "property" concept is, is to say that the "static" part is named a "field" and the instance part is named a "property", achieved by holding a reference to the "field" in the "property".

Do you also call them "properties" when applied to methods and types?

Also, ordinary concrete fields with storage could implicitly use StorageField as their defclass, so Str name would be equivalent to Str name defclass StorageField (or @StorageField Str name if that's how the syntax goes) and then super.get and super.set do make sense for field storage access.

I'm not a language guru by any means but so far I like stephen's idea the best. Brian's looks like some crazy complicated syntax in an otherwise clean(ish) language. I know there's a lot of fan vs scala talk going around but there are some things we want scala to beat fan at. ;)

Do you also call them "properties" when applied to methods and types?

No, I think of "operation". But the terminology is secondary.

Both a "property" and an "operation" could also be termed a "bound field" and a "bound method", or a "meta field" and "meta method". They consist of the object representing the singleton structure of the code (Field/Method) and the dynamic state of that property/operation (the object itself).

For the meta object of a class, the same applies. The "MetaType" consists of the Type and the object instance.

class Person {
  @MaxLen = 32
  Str surname
  @LogAccess
  Void callSecure()
}
mixin MaxLen : MetaField {
  abstract Int maxLen
  Void onSet(Str newVal) { ... }
}
mixin LogAccess : MetaMethod {
  Void onCall() {
    echo("Calling method $method.name")
  }
}

At compile time, the annotation mixins are implemented by a new class specific to that field/method.

One thing that is needed is a way to tie the assignment of the value (@MaxLen = 32) to the maxLen field in the mixin.

I've given this a lot more thought, and I'm thinking that the instance or "bound fields" aspect of this idea needs to be separated. My initial problem is how to use OO features on pods, types, and slots so that we can remove facets and symbols as they exist today and wrap up breaking changes. Today facets are purely static annotations on types/slots. The problem with instance meta-data is that it requires a parallel reflection API and creates a lot of confusion with the problem at hand. I'm actually thinking instance based meta-data should be handled with compiler plugins since it requires weaving additional state into the class itself.

So let's assume that this feature is now strictly about moving static meta-data facets into the pod, type, and slot classes. Effectively we want to remove the facet name map and just use normal field definitions:

slot.facet(@foo)    // old way
slot.field("foo"")  // new way
slot->foo           // new way 

Ideally I'd just like to make all the system facets virtual methods on Type, Field, and Method:

class Type
{
  virtual Bool isSerializable() { false }
}

class Field : Slot
{
  virtual Bool isTransient()  { false }
}

Facets effectively just implement a virtual or abstract method with a const object:

@isTransient=true Str someField
@isTransient Str someField // shortcut as before

@isSerializable class Person { ... }

Not sure about convention, seems like isX is the right convention which also lets us work around Java's limitation that you can't escape keywords like transient to use an identifier.

But it still seems ideal to also enable a type or slot to extend mixins:

mixin BuildTarget
{
  abstract Str description()
}

This is where I am not sure about syntax because we want a simple syntax that lets us both extend a mixin and define values for the slots:

Option A:

@BuildTarget { description="compile everything" }
Void compile() { ... }

Option B:

@BuildTarget
@description="compile everything"

In the end all the fields get flatten into an actual Type or Slot class so I kind of prefer option B myself.

Couple questions:

1. does everybody like this direction to remove symbols and put meta-data straight on the reflection classes themselves as normal fields? I really dig this design myself.
2. are there specific suggestions on the syntax and how to organize meta-data this way?

I really like option A better above; it feels like a "with" block to provide an initializer to a const field. Also it associates description with the right facet, in case there are several mixins.

I really like where this is going, turning symbols/facets into specialization of the Pod/Type/Slot metaobject hierarchy.

Makes me wonder about compile-time vs. runtime, static typing vs. dynamic typing. We've got good reflection now, which is a dynamic view of the static type system.

JodaStephen's post got me thinking about data binding being a tuple of property and watcher. That feels like a from-the-outside view of it: take this property, and this object that's interested in it, and put them together in a BoundProperty.

Brian's direction here feels like a from-the-inside view of the same thing: declare that this field is a property, which may be or must be bound to something that implements this facet type or mixin.

I'm still spinning in my head trying to understand this; I'm very sure though that it's important. We seriously need some form of bound properties, listeners, whatever; but I don't think we want to solve it the Java way -- that feels like too much from-the-outside, let the programmer tie it all together.

I really like option A better above; it feels like a "with" block to provide an initializer to a const field. Also it associates description with the right facet, in case there are several mixins.

The huge disadvantage with grouping it by type is that it doesn't reflect what really happens: that all the "facets" becomes fields on a single reflective class. For example consider this case:

mixin A { abstract Str usage() }
mixin B { abstract Str usage() }

Normal type inheritance rules would apply such that I should be able to use these types together as facets by merging their slot namespace:

@A
@B
@usage="foo bar"
Str someField

Edit: effectively what this does is create the following field definition:

class SomeField : Field, A, B
{
  const override Str usage := "foo bar"
}

We could still use a syntax closer to "option A":

Option B

@A
@B
@usage="foo bar"
Str someField

Option A2

@A, B { usage = "foo bar" }
Str someField

Or an example with a lot of facets Option B

@FirstWithALongName
@SecondWithALongName
@ThirdWithALongName
@attributeWithALongName = "some string ... yadda yadda"
@anotherAttributeWithALongName = 42
@whatever = true
Str someField

Option A2

@FirstWithALongName
,SecondWithALongName
,ThirdWithALongName
{
   attributeWithALongName = "some string ... yadda yadda"
   anotherAttributeWithALongName = 42
   whatever = true
}
Str someField

@Brian, you're right about option B being more like what really happens; I guess I'm thinking that it merges two concepts into one syntax: one (@A) is a declaration, the other (@usage...) is a definition.

helium's A2 option, where the mixins declarations are a comma separated list of names, and any definitions are inside a block, seems like it makes sense.

I don't know, the concept seems right, but I don't know about the syntax.

The problem with Helium's suggestion is how does it look like to just override a virtual method on type/slot itself, using that syntax would probably look something like this:

@{ isTransient=true }
Str someField

does everybody like this direction to remove symbols and put meta-data straight on the reflection classes themselves as normal fields?

No, I truly believe that this latest direction is the wrong one.

Firstly, I believe that having the facet info as a subclass of Field/Method/Type is very confusing. It imposes a link and constraints that don't need to be there, which results in the weird syntax issues being discussed above.

I strongly believe that the Field/Method/Type objects should remain as simple singletons representing the static structure of the code. This is the standard mental model, and avoids conflating two concepts.

Also, the mixin and generated subclass approach reduces the ability of each mixin to be properly accessed for more than just data. In the examples above, the generated subclass is simply a holder for one item of additional state. But I want facets to be real classes with real methods that act on their own state.

So, what direction would I go in?

@Timeout(10sec)
class Foo {
  @Transient
  @DBConnect("username", "password")
  DBConnection conn
}

The simplest way of looking at this problem is to say each facet is a class, not a mixin, and the Type/Slot holds a reference to the instance of the facet. The facet syntax is then a straightforward constructor call, preceeded by an @:

const class Timeout {
  const Duration anyNameYouLike
  const Str errorMsg
  Timeout(Duration dur, Str errMsg := "Error!") { 
       anyNameYouLike = dur; errorMsg = errMsg }
}
const class Transient {
  Transient() { }
}
const class DBConnect {
  const Str user
  const Str pw
  DBConnect(Str user, Str pw) { this.user = user; this.pw = pw }
}

Note how, as a standard constructor, we can use default parameters, and fromStr overrides without any extra rules.

The Type/Slot then simply has a list of objects which are the facets:

// option SJC-A
class Type {
  const Obj[] facets   // list of facets for that type
}
// setup code for Foo class load sets
Foo#.facets = [Timeout(10sec)]

// option SJC-B
class Type {
  const Type:Obj facets   // key facets by their type
}

// option SJC-A
class Slot {
  const Obj[] facets   // list of facets for that slot
}
// setup code for Foo class load sets
Foo#conn.facets = [Transient(), DBConnect("username", "password")]

// option SJC-B
class Slot {
  const Type:Obj facets   // key facets by their type
}

Note that although the Type/Slot does contain the facets, we haven't needed to do anything complex like create specialised subclasses. All the facets are just instances in a list/map on the single instance of Type/Slot.

Accessing these facets is easy using standard syntax:

// option SJC-A
Transient t = Foo#.facet(Transient#)
DBConnect dbc = Foo#conn.facet(DBConnect#)
Str user = Foo#conn.facet(DBConnect#).user

// option SJC-B
Transient t = Foo#.facets[Transient#]
DBConnect dbc = Foo#conn.facets[DBConnect#]
Str user = Foo#conn.facets[DBConnect#].user

// dedicated syntax?
Transient t = Foo#@Transient
DBConnect dbc = Foo#conn@DBConnect
Str user = Foo#[email protected]

This handles the current static facets simply and easily. By having facets as classes rather than mixins, there is no need to create a generated class that mixes them together (with the complexity and risk of inheritance clashes). The complex facet syntax discussed above also goes away - each facet is independent, and can have arbitrary methods (although I suspect they have to be const).

This whole approach can be extended at a later point (soon please ;-) to cover per-instance facets - there are many use cases for them. The only difference for per-instance is that it needs to be able to store facet information on a per-instance basis (as well as per-structure).

Note that my syntax choices leave open extension to per-instance facets:

DBConnect dbc = instanceOfFoo#conn@DBConnect

The simplest way of looking at this problem is to say each facet is a class, not a mixin, and the Type/Slot holds a reference to the instance of the facet.

This is my least favorite model, but has some strong advantages:

1. it is the model used by Java annotations and .NET attributes
2. probably the easiest to integrate for FFI
3. lets us use existing types and type literals to replace symbols

However, I've always thought that model was a pretty poor way to model meta-data. To me the primary use case is about named values, not instances. It is the name that is most critical, and when you use an instance the name is nebulous - is the name the type? is it any super type?

But I'm kind of thinking about a new direction now. I keep thinking about this problem trying to get my head around to look at it with the "right abstraction" (to create a truly general purpose feature). The way I'm approaching the problem now is this: what we are really trying to do is create dynamic maps where the keys are statically typed. This is kind of where the original symbol idea came from. Put another way, I don't necessarily know all the different "named things" that a type/slot might have, but I do want to make sure those "named things" are statically defined, typed, and use the namespace consistently.

Suppose instead of using symbols as we use them today, we approach them more as a cross between the . and -> operator:

Op  Name         Key            Result
--  -----        ---            ------ 
.   static call  identifier     return type of statically typed method
->  dynamic call str identifir  obj, determined at runtime
@   symbol call  symbol         determined by symbol type

Implementing how @ is mapped then becomes an API issue like sys::Obj.trap. It can be implemented by Pod, Type, and Slot for facets:

field@transient
obj.type@serializable

But we can also use it on objects in an application specific manner. Suppose localization is done via:

Locale.current@someKey

Or configuration is done via some utility class

config@someKey

It definitely seems like what we want is something between static and dynamic calls, but where I statically type the keys (maybe a bit like structural typing).

Ignoring syntax entirely for the moment. I like current Fan facets more than Java's annotations. Name/value pairs make more sense to me than unnamed attachments (though FFI friendliness is a good argument for doing things like other languages). The main question I see raised by Brian here is whether those name/value pairs are best as maps or as object/class fields.

I think the question (as pointed out by Brian) became especially interesting once those names became namespaced (with symbols) and the values became typed. They start sounding a lot like fields. So if they are fields, should you also allow ordinary named types, too? And what object are these fields on anyway? And so on ...

It opens a bit of a bag of worms (as we've seen here) but only because we discovered that asking this question made us wonder if we had really thought through the meaning of metadata in the first place.

I don't think we need to be afraid of thinking this through. We just need to make sure that the final decision is both as useful and accessible (to ordinary developers) as possible. FFI is also important in my opinion, but I wouldn't rank it the top objective. There could be other ways to interoperate with Java/.NET metadata if necessary.

As for the latest "symbol call" concept, I think it has some merit (and is related to my semi-request for extended symbol value type inference on maps and whatnot), but I also have concern about having too many parallel features.

I agree with that. It feels to me like this is coming down to a question of where exactly Fan will position itself among the languages.

Facets originally felt kind of like a compile-time static, configuration-of-build feature. Then use cases like localization came into the discussion, and I started thinking it was more of a compile-time declaration of types, load-time assignment of values.

Now I'm trying to understand how it fits with ...

In Smalltalk you get runtime access to your metadata: an object has a class, which is also an object; you can query and even reassign methods in a live system.

Javascript is completely wide-open, you can create a raw object and load it up with functions however you like.

In Fan we have static typing of metadata: Type, Slot, Field, Method are available at runtime but are read-only constant. We do however have a mechanism for declaring types as const, but loading them with different values: that was the whole with-block thing.

I'm thinking that concept should somehow apply to metadata: Type, and Field and Method, are const, but their slots can be assigned on creation using a with-block analog.

But what is "creation" for a metadata type? Normally you write source code, and the compiler "creates" the type. Which gives us compile-time configuration of metadata.

Some of the syntax discussed, brings that forward to load-time: a facet would declare a type and default value, and the system would look in various places for config files and use non-default initializers.

It seems to me that per-instance facets is the next step in that progression. In Smalltalk, or Javascript, you'd solve those use cases by dynamically modifying your object: swapping out slots for ones that do something different.

In Fan we don't want that (I hope we don't!); we want to keep our static type safety, along with our dynamic invokes for when we want them.

But, we do have that with- initializer I mentioned: how about using a form of that idea as a way to introduce during runtime a modification to type/slot metadata? Since our metadata is const, we couldn't go crazy with modifying methods out from under running code, but with this we could introduce a new object, of an existing Type but with an initializer to modify some of its metadata.

...I have a feeling I'm stopping just when I'm almost starting to make sense. At least I hope I got that far! I don't know yet how this could work, but I feel like it's something.

To me the primary use case is about named values, not instances.

Then perhaps we differ. I think named values are a degenerate case of types.

I see this discussion framed as a way to provide a form of compile-time aspect-oriented-programming. I'm looking to have a language feature such that adding a facet can fundamentally alter how the main object/field/method works. There is tremendous potential power available, yet with great safety due to the static typing.

I've been trying to decide on a good example. I'm going to go with a popup in a GUI which has an OK/Cancel button. The idea here is that you have a large object model that is collecting data from the user. As the user enters the popup, you want to store the data entered in the popup in a "safe" area, so that if the user hits cancel, the changes aren't applied, but if they hit OK then they are. There are lots of ways to achieve this, but I'm going to outline a facet based one with advanced facets:

@OKCancelSupport
class Person {
  Str? forename
  Str? surname
  Address address := Address()
}
@OKCancelSupport
class Address {
  Str? line
  Str? town
  Str? postCode
}
// main code
person := Person()
person.surname = "Smith"
person.town = "New York"
// enter popup
person#@OKCancelSupport.enter()
person.surname = "Jones"
person.address.town = "London"
// hit ok:
person#@OKCancelSupport.commit()
// person.surname == "Jones"
// person.address.town == "London"
// hit cancel:
person#@OKCancelSupport.rollback()
// person.surname == "Smith"
// person.address.town == "New York"

So, how do I envisage the magic working? Well, the person instance holds a reference to an instance of OKCancelSupport which is a real class with state and methods. Adding OKCancelSupport causes each field in the object to report changes to the instance. Once enter() is called, the original value for each field is stored as the value is changed. This applies down to Address too. When commit() is called, the original value store is thrown away. When rollback() is called, the original values are reapplied. (Again, there are variations on this design)

class OKCancelSupport : InstanceFacet, FieldAccessFacet, FieldListenerFacet {
  [Field:Obj?]? originalValues
  override Void onGet(Field field) {  // override FieldAccessFacet
    if (field@OKCancelSupport != null) {
      [email protected]()
    }
  }
  override Void onSet(Field field, Obj newValue) {  // override FieldListenerFacet
    if (originalValues != null && originalValues.contains(field) == false) {
      originalValues[field] = newValue
    }
  }
  Void enter() {
    originalValues := [:]
  }
  Void commit() {
    originalValues := null
  }
  Void rollback() {
    originalValues.each(Field field, Obj value) {
      field.set(instance, value)  // instance inherited from Instancefacet
    }
  }
}

BTW, this program is incomplete and untested - the aim is to communicate the concept.

The concept is that real, meaningful functionality can be abstracted out away from the main classes themselves. Its an evolution of OO that is very powerful.

Of course all the other ways of using facets are also supported - config, localization, flags (like transient). Its fully namespaced and typed, and less concepts to learn.

BTW, I'm deliberately ignoring static fields and methods in what I write. I strongly suggest that we only consider them at the end, as they are a degenerate case.

Brian, on your @ is a cross between the . and -> operator, I can't grok what that really means. It feels pretty complex, as I can't see how you'd make the keys statically known and namespaced, or the values typed. And obviously it doesn't provide the full power of intelligent metadata.

I'm looking to have a language feature such that adding a facet can fundamentally alter how the main object/field/method works.

That is what I am looking for too, but unless we create a parallel instance based reflection hierarchy it doesn't seem right to tackle this with facets. That is why I am thinking this sort of thing is better handled with a compiler plugin which would have the full power to weave new fields/methods into the AST at compile time.

No matter what there is some language level abstraction for pod/type/slot meta-data. The key question is how general purpose can we make that feature so that once you learn that technique you can apply it to your own APIs and classes. I am not satisfied with the Java/C# approach because it limits meta-data specifically to static structure and not instances.

Brian, on your @ is a cross between the . and -> operator, I can't grok what that really means.

I think it is pretty much the same thing you are trying to get at with your use of @ - a way to dereference some meta-data namespace on an class:

person#@OKCancelSupport.enter()

What you are saying is something along the lines of:

person.instanceType.facet(OKCancelSupport).enter()

What I am saying is that I can't grok a separate instance based reflection hierarchy, so I am trying to make accessing meta-data by symbols something any instance can implement:

[email protected]
person.getSymbol(@okCancelSupport).enter()

That example doesn't make sense though without true AOP because you have to weave so many traps/state into the class - which is why I think complicated things like might be best tackled with a compiler plugin.

Stephen do you think you have your head around your ideas enough to propose something concrete? Specifically I am interested in your thoughts on the instance based reflection model.

OK, here is an outline of my thoughts.

Person#    // Type, const
person#    // MetaType (or BoundType)

Person#surname  // Field, const
person#surname   // MetaField (or BoundField)

class MetaType {
  Type type
  Obj target
  Facet[] facets()
}
class MetaField {
  Field field
  Obj target
  Facet[] facets()
  Obj get()
  Void set(Obj newValue)
}

// current code
person.type
// new code
person#.type

Note that these MetaType/MetaField/MetaMethod objects are short-lived not long-lived. They are created on demand:

// new code
person#surname.get()
// implemented in low-level equivalent as
new MetaField(Person#surname, person).get()

(The MetaType concept is a little weird. It doesn't actually gain us a lot except consistency, but thats not a bad reason)

Now, extending to facets.

Given the above definition for OKCancelSupport I'd expect the following Java equivalent to be generated:

public class Person {
  private OKCancelSupport facet$OKCancelSupport
  public facet$OKCancelSupport() {
    if (facet$OKCancelSupport == null) {
      facet$OKCancelSupport = new OKCancelSupport(this);
    }
    return facet$OKCancelSupport;
  }
  ...
}

translating the call:

person#@OKCancelSupport.enter()  // Fan
person.facet$OKCancelSupport().enter()  // Java equivalent

I'd expect the either the Fan type database or the class to have a record of all the facets implemented by the type.

If the facet has no state and doesn't implement InstanceFacet, then I'd expect the implementation to default to be a simpler static form:

public class Foo {
  public static final Serializable facet$Serializable = Serializable.INSTANCE
  ...
}

The same idea extends to field/method facets.

If the facet extends certain mixins, then the type/field/method that has the mixin applied will be altered to send matching events:

// coded as
@MyFacet  // extends FieldAccessListenerFacet
Str surname

// implemented as
Str surname() {
  return facet$MyFacet().onGet(Person#surname, _surname)
  // the value of the internal _surname value is passed in for optimisation reasons
}

In summary,

• MetaType, MetaField and MetaMethod (or BoundType, BoundField and BoundMethod) are new classes that parallel Type, Field and Method but store a reference to their target instance
• facets are regular classes that extend Facet
• facets are implemented via hidden variables and methods on the target class
• when the class is built, all the facets are known, so the AOP-like addition of listeners can occur without excessive complication

There are still optimisations that could occur, but this is the basic outline.

Thanks for the detailed post Stephen. That is along the lines of what I was thinking for "parallel reflection hierarchy"

I guess the question is does this extra complexity make sense? The whole set of duplicate type, slot classes isn't sitting well with me. But then again symbols aren't setting that well either.

Brian, in your first post in this thread you talk about bound slots. That already implies a parallel hierarchy, because the methods are different depending on whether it is bound or not:

Field {
  Obj? get(Obj target)
  Void set(Obj target, Object? newValue)
}
BoundField {
  Field field
  Obj target
  Obj? get()
  Void set(Object? newValue)
}

It doesn't make sense to try and merge these two into one class.

Just want to note my current thinking...

I am not sure where we go from here. But I do think what we have today with symbols is better than where we were in 1.0.44, so we have made progress.

But I am not ready to make any changes yet, we need to think about this more.

However, it is extremely likely that facets/symbols/meta-data will go through another round of breaking changes before we lock things down. So please be aware of that if you are writing Fan code.

Extendable Types

I'm maybe a little slower than some a you guys at working through the implications of all this. Earlier on the idea came out, of being able to extend Field and Method, via inheritance.

Here I'm trying to think of it as if Type, Field, and Method were normal classes in Fan, and therefore declaring a class (or field, or method) becomes the same as declaring an instance of Type (or Field, or Method) and initializing its fields.

So class X { ... } is like X := Type { slots = ... }, and so on. This may not make sense to anyone else, but it seems to be helping me think about all this.

So anyway, with Field as an extendable class, you could maybe do this:

const class CheckedAssignmentField : Field
{
  // use super's Field.get;

  // override to check on set:
  override Void set(Obj? instance, Obj? value)
  {
    if (check(instance, value))
      super.set(instance, value))
  }
  // require to construct with a definition for check func
  Func check
}

trying to link that up with current facet syntax:

class A
{
  @CheckedAssignment { check = |,| { if (!isProperName(val)) throw... } }
  Str personsName
}

might be like...

A := Type   // normal type declaration
{
  fields =
  [
    CheckedAssignmentField : super(...) // normal Field init
    {
      // 'val' is the invisible, compiler-generated field
      // from the normal Field definition; and 'properName'
      // is some user-defined function somewhere
      it.check = |,| { if (!properName(val)) throw ... }
    },
  ]
}

now using an A

a := A.make { personsName = "Foo deBar III" } // checks if good name

The Build Context

For that equivalence ( class X {...} <=> X := Type { name="X";...} ) to work, you have to consider that there's a context in which the declarations live. "Normal" code is built by the compiler in the context of a pod. Scripts are given an unnamed system context. Web scripts might live in the context of a page. using statements import one context or parts of it, into the current context. For Fan code, Type/Slot/Field/Method are the interfaces exposed outside the context.

Okay, I need to think some more. I'm going to let this go out, even if it just shows that I'm still jumbled and confused. Still, I think there's something here, if only somebody could bring it to life.

@Kevin,

Some interesting ideas there. But I'd like to point out the issue is that we want a field to mixin multiple classes at one time - so it this is what makes it a bit different than normal OO, and more akin to AOP (Aspect Oriented Programming).

If we step back from type/slot meta-data for just a second, you can imagine that I might be able to declare a method parameter like this:

Void m(Alpha,Beta param)

That would say that param must implement both the Alpha/Beta types. Sure I could define a mixin that extended both of those types and force everyone to implement that type. But I might have existing code that already implements Alpha and Beta, but won't know anything about my new combo-type. It might get really interesting if we had structural typing.

It is this sort of "type mixing" that suites meta-data really well.

My response is just a bit of random mumbling - but I think if we keep slicing and dicing this problem from different aspects that something really good will emerge.

more akin to AOP (Aspect Oriented Programming).

I would say that there's still a big difference between anything here and AOP. In AOP, I'd say a fundamental concept is reaching in to modify the program from outside, such as with CSS selectors applying style/behavior to HTML documents. (CSS is very AOP, as I see it.)

As with HTML, when using AOP in a program you might annotate things to make them easier to select (like when using class or id attributes in HTML), or you might design a careful structure to be easy to query, but the thing making changes operates from outside.

Here, we seem to be focused on always directly stating in code what kinds of behavioral modifications should be applied to a slot, type, or pod, and I think that's different from AOP.

Good or bad depends on your perspective.

Here, we seem to be focused on always directly stating in code what kinds of behavioral modifications should be applied to a slot, type, or pod, and I think that's different from AOP.

Good point, although to me AOP is less about whether you do it up front or after the fact, and more about implementing cross-cutting concerns cleanly without cluttering up your code. So I would lump lots of these things into AOP:

• listener/notification of field changes
• persistence hooks into field changes
• security checks for field access
• auto-auditing of field changes
• trapping/logging of methods

Those all seem to be the same use cases we have been discussing here with bound types/field right?

It is this sort of "type mixing" that suites meta-data really well.

Why?

What benefits does this provide, given it adds greatly to the complexity. Why can't each annotation/facet be independent of one another.

On AOP, I think that if the basic compile-time mechanism is got right, there is no reason why extra facets can't be specified at load-time (probably causing an extra compile-step).

What benefits does this provide, given it adds greatly to the complexity. Why can't each annotation/facet be independent of one another.

I think we are saying the same thing. For example if I want to annotate something as transient, compiled to js, and with a listener I should be able to do this:

@transient @js @listener Str foo

I don't want to have to require this:

@TransientJsListener Str foo

class TransientJsListener : Transient, Js, Listener { ... }

I don't want to have to create a single OO class for each combination of facets - that was what I was trying to say.

I don't want to have to create a single OO class for each combination of facets

Agreed.