This example code:

mixin Tree
{
  abstract Tree? parentNode
  abstract Tree[]? childNodes

  virtual Void addChild(Tree child)
  {
    child.parentNode = this
    if (childNodes == null) childNodes = [,]
    childNodes.add(child)
  }
  //...
}

class MyTree : Tree
{
  override Tree? parentNode
  override Tree[]? childNodes

  MyTree? parent { get { return parentNode as MyTree? } }
  MyTree[]? children { get { return childNodes as MyTree[]? } }


  static Void main()
  {
    MyTree root := MyTree()
    root.addChild(MyTree())

    echo(root.childNodes)

    echo(root.children)

  }
}

generates this output:

[fan.bug2_0.MyTree@47393f]
null

and I don't know why. I'm trying to make an accessor for the mixin's data fields that can be used without casting. But, I'm getting null back from the children accessor method.

If I change the accessor to:

MyTree[]? children { get { return (MyTree[]?)childNodes  } }

then I get the output I want:

[fan.bug2_0.MyTree@47393f]
[fan.bug2_0.MyTree@47393f]

So, no big deal. But I don't get why the as isn't doing what I thought it would.

If you dump the type of childNodes you will see it is sys::Obj?[] since it is allocated with the expression [,]. Since Obj[] is not a subclass of MyTree[] so it is the correct behavior to return null. The thing here is that the type of the list is what matters, not necessarily what is inside it.

Without knowing exactly what you are doing, I tend to tackle these sort of problems with covariance like this:

abstract class Base
{
  abstract Base parent()
}

class MyBase : Base
{
  override MyBase parent
}

Side note: using a nullable type for the as expression is implied, I am going to make that a compiler error.

Okay... so when I used the as it was giving null to report that the array isn't of that type. But when I used the cast operator it was coercing the array to the type, and since it only held elements of the correct type, the cast worked. Correct?

I had been thinking that as was just another syntax for the cast operator.

Btw the example above just comes from making a covariant accessor for the non-covariant mixin field.

Okay... so when I used the as it was giving null to report that the array isn't of that type. But when I used the cast operator it was coercing the array to the type, and since it only held elements of the correct type, the cast worked.

This behavior surprises me. I would expect as to return null only in the cast that casting would throw a cast exception.

I changed the compiler to not allow a nullable type to be used with the as expression.

Okay... so when I used the as it was giving null to report that the array isn't of that type. But when I used the cast operator it was coercing the array to the type, and since it only held elements of the correct type, the cast worked. Correct?

For most types, things work like this:

// Fan code      emitted Java code
(Foo)x           (Foo)x
x as Foo         x instanceof Foo ? (Foo)x : null

But Lists, Maps, and Funcs work a little different since the Java type system doesn't actually capture their types:

A cast to List/Map/Func type is basically a no-op:

(Str[])x

In the example, we don't actually walk the list to check each item. That kind of sucks because you might get a type error when you actually try to use the list later. But given that Fan does a lot of auto-casting, we definitely don't want to be walking the collection for type checks.

In the case of is and as operator for generics, we actually generate this code:

x is Str[]            // Fan code
x.type.fits(Str[]#)   // what really gets generated

Hope that helps explain what is really going on under the covers.

I still expect a CastErr for normal casting if as returns a null. If it can be determined in one case, why not the other?

I still expect a CastErr for normal casting if as returns a null. If it can be determined in one case, why not the other?

Using the current design (where a generic cast is basically a no-op), then the matching behavior for as with a generic type is to evaluate never to null (assuming correct base type of List, Map, or Func).

That makes sense to me for consistency. If there is consensus around that behavior I will change it.

In the example, we don't actually walk the list to check each item. That kind of sucks because you might get a type error when you actually try to use the list later.

I wonder if it would be feasible to store the type of the list in the list itself, e.g., list.elemType, in which case add could check that the new element fits the element type, and the compiler could do runtime checking of casting from one type of list to another type.

In any case, like Tom, I expect a CastErr in the exact case that as returns null. It's really surprising this isn't how Fan works, so if you want to maintain the existing behavior, I suggest making it crystal clear in the documentation for as and casting.

I wonder if it would be feasible to store the type of the list in the list itself, e.g., list.elemType, in which case add could check that the new element fits the element type, and the compiler could do runtime checking of casting from one type of list to another type.

The List does know its value type. But it seems really expensive to check every add, and in three years of programming in Fan that has actually never caused an error that I've seen.

But here is the tricky problem:

objs := Obj[,]
objs.add("a").add("b")
strs := (Str[])objs

Today, strs === objs, and would still report Obj[] as its type. We could potentially walk objs, check that everything is Str and return a new list typed as Str[]. But that seems awfully expensive.

In any case, like Tom, I expect a CastErr in the exact case that as returns null.

Yeah, I agree with you and Tom that the current behavior is inconsistent and should really be changed.

I agree, current behavior is surprising, and I think as and cast should be consistent with each other: CastErr when as returns null, seems right.

If List is really always Obj[], then it should be always safe to cast it to SomeType[], since SomeType always derives from Obj and can therefore be put in any list. Allowing as SomeType[] lets me use the compiler to ensure that the list only holds what I want it to. So, List as AnyType[] should always be non-null, I'm thinking.

This rule does seem right except when the variable being casted is null.

Obj? x := null
Str? y := x        // x is null, so casting is legal in this case
Str? z := x as Str

So while as returns null whenever a cast would throw a CastErr, the reverse scenario is not necessarily true.

But it seems really expensive to check every add

In Java this is an identity comparison; e.g. x.class == elemType.class, and therefore quite fast (as fast as two dereferences and a conditional can be).

We could potentially walk objs, check that everything is Str and return a new list typed as Str[]. But that seems awfully expensive.

I like the check for all elements, because it locates any potential problems to where the cast is first performed -- as opposed to 10 function calls later, when an object from such an array is used in a way incompatible with its type.

However, you couldn't create a new list, because then changes to the original would not be reflected in the new list, which would break the semantics of casting.

So, List as AnyType[] should always be non-null, I'm thinking.

On second thought, after seeing Brian's example, maybe that doesn't make as much sense.

objs := Obj[,]
objs.add("a").add(1)     // okay
strs := objs as Str[]    // now what?  

CastErr ideally.

In Java this is an identity comparison; e.g. x.class == elemType.class, and therefore quite fast (as fast as two dereferences and a conditional can be).

It isn't quite that simple. For example a list of Num can store Int, Float, or Decimal - I follow Java's contra-variant array model here.

So we have to check subclasses. In many cases we can utilize Class.isAssignableFrom which I understand HotSpot optimizes to be near the performance of a instanceof operation. However since we can't use the Java type system to capture generic types like Int:Str, in the end we would have to route to Type.fits which will always be pretty slow because I have to walk the inheritance tree myself in Java/C# code.

I don't really love the current solution, but I do think it utilizes the best trade-off of compile time safety with runtime performance. Basically Fan isn't much different than what type-erased-generics are doing in Java. Although I think Fan is better because at least the generic types maintain their parametrization for reflection (even if they don't use them for runtime type checking).

@KevinKelley:

objs := Obj[,]
objs.add("a").add(1)     // okay
strs := objs as Str[]    // now what?  

As JohnDG said, ideally it would throw a CastErr. But practically speaking, as a semi-dynamic language, there isn't much we can do until someone attempts to get an item as a Str and gets a CastErr. That isn't ideal because the exception doesn't occur where the actual problem is. But I still believe it is the right trade-off because that sort of error is extremely rare, where auto-casting Lists/Maps is extremely common.

objs := Obj[,]
objs.add("a").add(1)     // okay
strs := objs as Str[]    // now what?  

This may be a stupid idea but I thought I'd throw it out there: how about wrapping the target of the as expression in a special list that simulates the as semantics on each element during traversal?

class AsList : List
{
  V[] target
  Type of

  AsList(V[] target, Type of)
  {
    this.target = target
    this.of = of
  }

  Void each (|V?, Int| func)
  {
    target.each |val, i| {
      func(of.fits(val) ? val : null, i)
    }
  }

  ...
}

On second thought, that would weaken the result type of the as expression. That is, the expression strs := objs as Str[] would yield a Str?[]? instead of the expected Str[]?.

This may be a stupid idea but I thought I'd throw it out there: how about wrapping the target

As JohnDB pointed out the result of an as expression must reference the original instance. That is this axiom must apply:

r := x as T
if r != null then r === x must be true

That is this axiom must apply ...

Unless it's an Int, Float, or Bool being unboxed? Well, I guess if it was an Obj (or Num) before, and it stayed boxed (since as gives nullable results), maybe not an issue here.

Just thinking about where === expectations might break. I guess normal casting (to plain Int rather than Int?, say) could break them.

Just thinking about where === expectations might break. I guess normal casting (to plain Int rather than Int?, say) could break them.

The compiler will report an error if you attempt to use the as operator with a value type (so that case is already covered).

Is everyone comfortable with the trade-offs I have made regarding casts of generic types? If the compiler knows that the cast will fail it will report an error - for example casting Str[] to Int[] will never be allowed. Otherwise if the cast could succeed the compiler auto-casts, but the runtime does not check it.

I haven't done a lot of work with Java generics, but I assume they work the same way.

Assuming we agree to that behavior, then I propose to change the as operator work consistently with casting for generic types. Since the cast operator always succeeds with a generic type (assuming the correct base type), then the as operator will always return non-null.

I agree; that seems best.

I'd lean towards making it fail in both cases (if it's not a huge performance issue), but I think consistency is better than not, so I agree that allowing it to succeed in both cases is better than the current behavior.

I pushed a fix to make as work like cast. Note that is still actually calls Type.fits, but I think that is that the correct behavior:

fansh> Obj o := Obj["a", 3]
[a, 3]
fansh> o as Str[]
[a, 3]
fansh> o is Str[]
false
fansh> (Str[])o
[a, 3]

Seems right to me: can use strongly-typed lists if I want, or use them as loosely-typed collection of Obj if I want.

wait - does that mean I would be expecting an Int from a Str[]?

Here is parallel Java code for this problem that pretty much works just like Fan does:

ArrayList list = new ArrayList();
list.add("foo");
ArrayList<Integer> x = (ArrayList<Integer>)list;  // this works
System.out.println(x.get(0).intValue());          // fails here

Although I notice that Java won't let you cast from List<Object> to List<Integer>.

Although I notice that Java won't let you cast from List<Object> to List<Integer>.

This is actually a major annoyance, because the Java type system is not powerful enough to express some relationships, which means the ultra conservative generics rules often just get in the way and prevent perfectly good code from compiling.

Could we have a method List.checked() so that we have the option to choose type safety over performance?

Could we have a method List.checked() so that we have the option to choose type safety over performance?

I don't mind adding a new method to List, but it will have to be something you call explicitly (the compiler won't call it).

Today you could do something like:

list.all { it.type.fits(Expected#) }
list.findType(Expected#).size == list.size

We could add something like this which returns true if all the items fit a specific type:

Bool allType(Type t)

Could we have a method List.checked()...

Personally I'd prefer a SafeList that checks on addition, because it's most helpful to get the error at the point of insertion, rather than much later when a client decides to call a checked method.

I don't mind adding a new method to List, but it will have to be something you call explicitly (the compiler won't call it).

Agreed.

We could add something like this which returns true if all the items fit a specific type:

I'm not sure we need to go that far. Just wrap the list in an instance that checks element type on insertion. This is all that java.util.Collections.checkedList() does--it does not verify the elements already present in the list. If you're really worried about type safety you can always call checked() at the time the list is constructed.

The two cases I see for using this API are:

1. Some buggy code somewhere in the program is adding an element of the wrong type. A bug like this is generally hard to track down because the error doesn't surface until later when you try to cast the bad element to the expected type. In this scenario checked is used to make the error surface immediately.
2. You are passing a List to some outside code that is going to change the list's contents, and you want to guard against the outside code doing bad things to your list.

I'm sure there are other uses but these are the two main ones I can think of.

In the same vein I think we could also add Map.checked() (as well as List.isChecked() and Map.isChecked()).

Personally I'd prefer a SafeList that checks on addition, because it's most helpful to get the error at the point of insertion, rather than much later when a client decides to call a checked method.

I think we're on the same page, except I'm calling it "checked" and you're calling it "safe."

Just to clarify, what I'm suggesting is similar in semantics to List.ro() except that you're getting a type-checking view instead of a read-only view.

@qualidafial - do you mind creating a new topic to propose and discuss this feature?

Done.

http://fandev.org/sidewalk/topic/562

Although I notice that Java won't let you cast from List<Object> to List<Integer>.

Yes, I forgot - Obj as type parameter in Fan is the Java equivalent of raw type.

It does get a bit of time and practice to get used to, but once you are there it is actually quite natural.

I'm thinking of some sort of compiler warning here (like the unchecked warnings that you get for casting List to List<Integer>), but then the dynamic nature of Fan won't make much sense this way...

All in all, I'm happy with where things are right now :-)

P.S. List<Object> is not a raw type, so the cast will fail.