I have noticed a lot of Fan vs Scala lately. In some ways I guess that is to be expected since both languages are statically typed alternatives on the JVM (unlike the dynamic languages such as Groovy, JRuby, Jython, and Clojure). But just remember that we are all on the same team trying to advance the state of JVM languages - we share the common goal of convincing Java programmers that there are alternatives to the Java language itself.

I think Scala is a really great language and I'm always on the lookout of good stuff to steal from it. But I do believe there are some key philosophical differences which I haven't really seen talked about. Note I haven't used Scala to write any production code, so I am not an expert by any means - but I do try to keep up with it.

Both Fan and Scala are trying to take software scalability to the next level, but I think we are taking different paths to get there. Scala's focus is clearly on a sophisticated type system which can catch as many errors as possible at compile time. It is also focused on mixing functional style with OO style (more so than Fan which only uses functional programming primarily as an enabler of better APIs).

Fan uses a static type system also - but as just one tool in the toolbox. Rather much of Fan's effort has been focused on overall architecture - more of a "cathedral" approach to providing a unified solution for all the different pieces required to build, test, and deploy software (yes I know IDE solutions are lacking, but I think this will take care of itself with time). This philosophy on the entire platform and the end-to-end programming experience versus just the language is reflected in Fan's features and even this site itself.

Pods

Fan has a built-in module system called "pods" that is seriously baked into the core. This enables a simple, consistent namespace of types based on pod::type. Pods are intertwined into Fan's reflection APIs to enable all sorts of things that are impossible to do in Java (and I assume Scala) such iterate all the installed types:

Pod.list.each |pod| 
{ 
  echo("--- pod $p.name $p.version ---")
  pod.types.each |t| { if (t.isPublic) echo(t) }
}

If I were to pick the absolute most important feature required for scalable software, I personally would pick a first class module system. I think this is a key difference between Fan and Scala. Scala follows more of the Java model of packages, with an unspecified module system. From what I can tell, OSGi seems to be the preferred Scala solution for modularity, but it isn't integrated tightly into the language like Fan.

Slot Reflection

One of major design decisions Fan made a long time ago was to disallow method overloading. Method overloading is the kiss of death for easy reflection and meta-programming. In Fan any slot can be identified with the qualified name pod::Type.slot. Slot literals are built into the language:

m := List#join  // same as Slot.findMethod("sys::List.join")

Languages like Java or Scala which support method overloading require the parameter types in order to perform method reflection. Identifying methods strictly by name is also a key enabler for efficient dynamic dispatch. This is a trade-off Fan and Scala have made differently - Fan took away a Java feature in order to simplify reflection and dynamic dispatch.

Type Database

Having a built-in module system enables all sorts of other really awesome features. One of the most important is Fan's type database. Because Fan knows about all the installed modules and each module's types, we can index the installed types to provide a unified solution for reflecting things like plugins. For example, in Fan to query the handler for a given URI scheme:

UriScheme scheme := Type.findByFacet(@uriScheme, "file").first.make

More Dynamic Type System

The example above illustrates the "auto-cast" of Fan. Typical Fan code uses reflection like this a lot, and this sort of meta-programming style does not lend itself to static type checking. This is why Fan will let you use Obj as a wildcard whenever a more explicit type is expected. Note the wildcard feature does not really weaken the static type system much, because it typically only kicks in with reflection APIs (which would have have required an explicit cast anyways). As an example, I've just performed a huge amount of refactoring on the code base for symbols and virtually everything was caught at compile time. But this is a case where Fan's type system makes a trade-off towards a weaker static type system, but easier for meta-programming and reflection.

Fan also supports the "->" operator to perform a dynamic call. I know you can do similar things with Scala's support for arbitrary method names, but it isn't quite the same as building it into the language. This operator is designed to play nicely with Fan's type system via auto-casting. More importantly it is a built-in language construct that we can map to the new invokedynamic opcode on the JVM. I don't use this operator a lot myself in production code because I prefer the safety and performance of static calls, but I use it a ton in my tests - I couldn't live without it.

Build Engine

Fan includes a built-in engine for writing build scripts as Fan scripts. Sure it is super simple and doesn't have all the bells and whistles of other Java-land solutions. But it follows the Fan philosophy of having everything ready to use and tightly integrated into Fan's overall architecture. Build scripts leverage Fan's declarative style, but the ability to easily write a new method when you need it. This is opposed to the Java-land way of doing declarative stuff in XML, and then having to write functions separately (something that I've always found awkward).

Furthermore much the build tooling is available via the reflection APIs to easily create new tools. For example to get the fandoc for a method and format as HTML:

fandoc := Str#split.doc
FandocParser().parseStr(fandoc).write(HtmlDocWriter())

Immutability

Fan's type system is by no means as flexible as Scala's. But I do think the notion of immutability is more hardwired into Fan via the notion of const types and const fields. The notion of immutability is also baked into all the reflection and concurrency code. This makes it almost impossible to share mutable state between actors (threads). The only way to share mutable state is via the Unsafe class which exists only as an escape hatch.

Perhaps the prototypical example is the Obj.isImmutable method - the ability to reflectively query any object's mutability.

Portability and JavaScript

Fan started life as portable language for both the JVM and .NET CLR. That forced us to create a clean separation between Fan code and the underlying platform. It turns out that having that underlying philosophy in the architecture is paying huge dividends as we work to make Fan code run seamlessly in the browser by compiling to JavaScript. Unlike GWT which is a separate solution from Java, Fan's JavaScript support is being integrated into the whole experience including tooling and deployment. There is one portable system, graphics, and widget library that just works everywhere.

Some might argue that Fan's attention to portability is a weakness compared to Scala's focus on the JVM. It is certainly a trade off, but I think in the end both Fan and Scala will make for first class JVM languages. And I think Fan's portability will really pay off in the long run.

APIs

Because of Fan's attention to portability, we're trying to create clean, elegant core system APIs required by most typical apps. This lets us define APIs that use Fan idioms such as closures, immutable, and serialization. But it also lets us really focus on an awesome API experience. Fan is much more about its APIs than the language itself - the language is designed to support the APIs, not vise versa.

True, Fan will never have the breadth of APIs available Java - but those are available via the Java FFI. But by focusing on common APIs in Fan such as IO, DateTime, Weblets, etc, Fan code shares common definitions for key abstractions. I think Fan and Scala differ here quite a bit. Scala tends to use implicit conversion - for example to map from plain Java String to rich string. Fan attempts to normalize core concepts like strings, DateTime, IO, etc into powerful, centralized classes which don't require conversions.

Serialization

One of Fan's pivotal features is its human readable JSON like serialization syntax. This format supports Fan's type system (which is predicated on a clean module/namespace infrastructure) so that object trees can be round-tripped via IO with no loss of fidelity. This serialization infrastructure is widely used in all sorts of features such as facets, symbol configuration, messaging between actors, etc. For example to read an list of ints from a string:

"[0, 1, 2]".in.readObj

When you look at typical Java system architectures, there are huge amounts of wasted effort which go into things like XML configuration files and solving all the associated impedance mismatch that follows. The notion of declarative configuration in Fan is best illustrated with the upcoming symbols feature. Now when you have a system wide setting, you just create a symbol:

pod acme
{
  ** Some fandoc describing symbol - type inferred as Duration
   virtual timeout := 10sec

  ** List of server host names - type inferred as Str[]
   virtual servers := ["localhost"]
}

When you want to use the symbol in your code, you use a symbol literal which provides a statically typed value:

socket.options.receiveTimeout = @timeout.val

Then you can override the symbol's value at deployment time using a configuration file:

// {repo}/etc/acme/pod.fansym config file 
timeout=30sec
servers=["alpha", "bravo", "charlie"]

Everything is integrated in Fan's module subsystem, static type system, reflection, and documentation. This feature is also used for statically typed localization. It is precisely this sort of source-to-deployment model that we are really trying to make a great experience in Fan!

(Side note - this feature is only possible to efficiently implement with caching because we have such a comprehensive approach to immutability).

Conclusion

I don't particularly like Fan vs Scala debates, and my intention is not to diss Scala. But I did want to shed some light on my philosophy of Fan and how I think we have made different trade-offs. A lot of it boils down to Scala's focus on its static type system versus Fan's focus on modularity and meta-programming. There is no right or wrong, just different trade-offs that will suit different tastes. There is also definitely an element of cathedral versus bazaar - Fan is trying to bake more into a unified whole (which is something I can understand might not appeal to those who favor the bazaar model of gluing lots of pieces together).

I know the Fan vs Scala debates are probably only just beginning. I would just ask that those of you who participate (especially on this site) keep the conversation professional and constructive. Both Scala and Fan are pretty awesome technologies (if I saw so myself :-).

Generally, I think this was a very good Fan vs Scala article (probably the best I've read), focusing primarily on Fan's strengths. There are a few things I would like to point out though.

First, you entirely glossed over the issue of Java interop. Fan can use any Java library (more or less), but it has to do it through a separate FFI. In Scala, Java libraries are Scala libraries. There is no disparity. This has led to some unfortunate compromises (like null), but on the whole I think it was the right decision. The interop is the language, particularly on the JVM or the CLR.

Scala's JSON support is almost as good as Fan's. Scala does have a built-in library for JSON, but it isn't as good as literaljson. You should read a bit about the library, it's very good, certainly good enough to qualify as "human readable".

Fan's hybrid-dynamic type system has been a point of philosophical contention for a while now, but I would like to specifically take umbrage at your statement that "automatic casting doesn't weaken the type system". What you have done is add the following rule to the type system:

t : T2
T1 < T2
------- T-AutoCast
t : T1

That's in addition to the pre-existing subtyping rule (otherwise known as upcasting):

t : T2
T2 < T1
------- T-Sub
t : T1

In other words, we can in two steps go from any type to any other type. This renders the type system all-but meaningless in the hands of a developer who isn't paying attention to such things. I'm in favor of powerful language features, but I don't think this is a feature that I can support.

Finally, I'm not sure how you can seriously claim that overloading impairs performance. I've written a few compilers in my time, and even in the most naive implementation, dynamic dispatch to an overloaded method is absolutely no slower than to a non-overloaded method. It just takes a little bit of name mangling and away you go (see: C++). HotSpot is hardly a naive compiler, and I can assure you that in practical tests, overloading has no appreciable impact on performance.

To put it bluntly, if you're seeing performance issues specifically because of method overloading, then you're doing something wrong with dynamic dispatch.

Coming down off my high horse: I would like to restate that I liked your article. Obviously, your bias would be toward Fan (as mine would be toward Scala), but that didn't impair a well-reasoned and positive comparison. Nice job!

Finally, I'm not sure how you can seriously claim that overloading impairs performance. I've written a few compilers in my time, and even in the most naive implementation, dynamic dispatch to an overloaded method is absolutely no slower than to a non-overloaded method. It just takes a little bit of name mangling and away you go (see: C++).

I think you misunderstood Brian's claim. Quoting:

Method overloading is the kiss of death for easy reflection and meta-programming.

He's only talking about reflection and meta-programming, not performance in general. Nobody here would argue that method overloading impairs the performance of statically compiled code. However when you introduce reflection into the picture, it's a different story.

In Java (and presumably in Scala too, correct me if I'm wrong) you can have multiple foo methods, each with a different signature. So at runtime when you call:

clazz.getMethod("foo", new Class[] { Bar.class, Baz.class })

The Class.getMethod implementation searches for all methods that fit those argument types, and attempts to decide which one is the best fit. Naturally this step incurs a performance penalty.

However in Fan every field/method must be uniquely named. So reflecting that member is as straightforward performance-wise as a hashtable lookup. I believe this difference is the basis of the "kiss of death" claim.

It would be interesting to benchmark reflection-based serialization in Java vs Fan. I'd like to see how well XMLEncoder/XMLDecoder stack up against Out.writeObj/In.readObj, respectively.

Aside: Here's a gem from Class.getMethod javadoc (emphasis, editorial commentary mine):

If more than one such method is found in (step) C, and one of these methods has a return type that is more specific than any of the others, that method is reflected; otherwise one of the methods is chosen arbitrarily. <headdesk/>

@djspiewak

First, you entirely glossed over the issue of Java interop. Fan can use any Java library (more or less), but it has to do it through a separate FFI.

Does that really matter? All the FFI does is specify a special syntax in the using statement. The nice thing about that is that it is clear if you are importing from Java or .NET. I assume that in Scala on .NET you use the exact same syntax when importing .NET namespaces as you do as importing Java packages. I guess my personal taste is that explicit is better.

Scala's JSON support is almost as good as Fan's. Scala does have a built-in library for JSON, but it isn't as good as literaljson.

I checked it out - pretty cool. Although Fan's serialization syntax goes way beyond just JSON. When you serialize a foo::Bar, you get back a foo::Bar. For example:

Sys.out.writeObj(GridPane
  {
    Label { text="Hello"; font=Font("12pt Arial") },
    Label { text="World!"; fg=Color.green }
  }, 
  ["skipDefaults":true, "indent":2])

Actually prints:

fwt::GridPane
{
  fwt::Label
  {
    text="Hello"
    font=gfx::Font("12pt Arial")
  },
  fwt::Label
  {
    text="World!"
    fg=gfx::Color("#00ff00")
  },
}

That is a chunk of text you can output from a GUI builder and can pass straight to the browser/SWT and have it deserialized into a widget. The format is tightly integrated into the language and all the APIs and supports the same using imports and type inference as the language. On the flip side, any serialized object is a valid Fan expression too.

but I would like to specifically take umbrage at your statement that "automatic casting doesn't weaken the type system".

Actually I said "doesn't weaken the type system much" - I totally agree it weakens the type system. The question is does it cause any practical harm in day to day programming? My experience has been no, most things still get caught at compile time. Like all engineering decisions it is a trade-off.

To put it bluntly, if you're seeing performance issues specifically because of method overloading, then you're doing something wrong with dynamic dispatch.

As qualidafial said, my point was only related to reflection of method lookup using a single string key. Even if you hash the types in your lookup, you still have to hash the types and then you can't have a simple method literal syntax. But to actually do a dynamic dispatch is not a simple hashing of types, it requires finding the best fit. For example:

class A {}  class B {}  class C : A {}

void foo(A)
void foo(B)
void foo(Obj)

obj->foo(new C())  // which version of foo gets called?

You actually have to examine the types of the parameters to find the best fit - I don't think you can do that with simple name mangling.

but that didn't impair a well-reasoned and positive comparison. Nice job!

Thanks for the feedback!

In other words, we can in two steps go from any type to any other type.

You only need 1 step in Scala: asInstanceOf. :-)

Auto-casting in Fan is useful precisely at the points where a programmer would insert a manual cast. But auto-casting has the advantage of cleaning up the code a lot. Fan code is much easier to read than Java or Scala.

For the particular language that Fan is, auto-casting is indeed the right tradeoff, and I don't believe it's significantly less safe (in day to day programming) than manual casting.

Now, the surest way to prevent cast errors is to forbid casting (ala Haskell), but to do that, you need a seriously strong type system. Fan isn't trying to be that. Scala is, but it's littered with junk like casting and null and other stuff.

I guess I should apologize for my blog posts if they've been troublesome. Still, thanks, Brian, for giving your opinions on what you think matters most. It's helpful to see your perspective, because it gives context.

@qualidafial

I think you misunderstood Brian's claim.

I think I did too. Very sorry, Brian!

@brian

Does that really matter? All the FFI does is specify a special syntax in the using statement. The nice thing about that is that it is clear if you are importing from Java or .NET. I assume that in Scala on .NET you use the exact same syntax when importing .NET namespaces as you do as importing Java packages. ...

I think it does matter. Interop is only as powerful as it is integrated. As I mentioned, in Scala, there is no disparity at all between Scala and Java: the two are essentially identical. This means that calling out to Java doesn't require speaking a separate language -- or, more specifically, using a separate language feature. Scala's Java interop is literally seamless, in every sense of the word. Fan on the other hand has a very good Java FFI, but it almost slaps you in the face with the fact that you're "shelling out" to another language. It works, but it's not as nice as I would like.

... I guess my personal taste is that explicit is better.

I guess we differ there, at least on matters of interop. :-)

@JohnDG

You only need 1 step in Scala: asInstanceOf. :-)

But it's explicit. That's the point I'm trying to make. Consider the following Fan pseudo-code:

Object bar := foo
AnyType baz := bar

In this code, foo can be of any type, and we've just implicitly mangled it into an arbitrary, and potentially unrelated type. This is assuming of course that Fan actually enforces casting within a hierarchy. If not, then we can simplify even further:

AnyType baz := foo

The type theorist in me is just cringing at this point. I can see the benefit to auto-casting in cleaning up cast-heavy code. For a language like Fan which rejects generics, such a feature would almost be a necessity. However, I really, really prefer to just avoid casting altogether. To me, casting is a sign that I was lazy in modeling my type signatures.

Now, the surest way to prevent cast errors is to forbid casting (ala Haskell), but to do that, you need a seriously strong type system. Fan isn't trying to be that. Scala is, but it's littered with junk like casting and null and other stuff.

Not really true. I almost never cast in Scala. When I do, it's usually just to make some stupid little prototype code work, and I'll come back to it later. Casting is the developer saying that "they know better than the type system". Well, with a really powerful type system (like Scala's), that almost never has to be the case. As long as you are using that type system appropriately, then all of that nasty casting goes away without any loss of type safety. Examples:

• Higher-kinded Map
• Using that map to avoid casting

In this code, foo can be of any type, and we've just implicitly mangled it into an arbitrary, and potentially unrelated type. This is assuming of course that Fan actually enforces casting within a hierarchy.

Fan does enforce casting within a hierarchy.

The point that Brian and I make is that the contrived example you show above doesn't occur very often in real-world coding.

The feature is used where a developer would just be inserting a manual cast anyway. This is why there is no significant loss of type safety.

What is the difference between:

Object bar := foo
AnyType baz := bar

and

Object bar := foo
AnyType baz := (AnyType)bar

aside from that one requires more typing than the other (as well as containing duplicated information on the typing of baz)?

I do grant there's one real danger of auto-casting: it can make developers lazy and just choose to use Obj everywhere (I've seen this in some Fan APIs), and to the extent that happens, static type safety takes a serious blow.

Not really true. I almost never cast in Scala.

Well, I never cast in Haskell. So there. :-)

As long as you are using that type system appropriately, then all of that nasty casting goes away without any loss of type safety.

My personal claim (without formal studies to prove anything) is that full static type safety provides little ROI as long as the runtime also provides dynamic type safety. Note that I do like some enforced type safety. A certain amount makes things more readable and improves performance and toolability.

Pure static and pure dynamic are both fine design decisions, too. But I think some folks see something philosophically wrong with taking a carefully planned middle road. I think a middle road can be fine, personally.

And without formal studies (and often even with), a lot really does come down to personal experience and opinion.

I do grant there's one real danger of auto-casting: it can make developers lazy and just choose to use Obj everywhere

I think that has more to do with lack of general-purpose generics than it has to with autocasting. I think some super lax generics could be nice, but I know that's not going to happen in Fan, so I don't fight it.

@JohnDG

Well, I never cast in Haskell. So there. :-)

:-) Fair enough. I still think that auto-casting only serves to make a dangerous language feature too convenient, but that may be more a personal opinion than an empirical fact.

@tompalmer

My personal claim (without formal studies to prove anything) is that full static type safety provides little ROI as long as the runtime also provides dynamic type safety.

I'm not convinced. Going from Java to Scala, I have found so many situations where I have saved literally hours because my code was correct by construction. Even really simple stuff like the following:

def sum(ls: List[Int]): Int =
  if (ls.empty) 0 else ls.head + sum(ls.tail)

This bit of code is quite respectable, and it's even more concise than the following:

def sum(ls: List[Int]): Int = ls match {
  case hd :: tail => hd + sum(tail)
  case Nil => 0
}

The key advantage to the second version is that the type system will statically guarantee that we are never going to try to access the head of an empty list. That's powerful. It's not hard to imagine how this advantage can be generalized to less contrived tasks.

For the record, I would never implement a sum function as given, it just serves as an example. :-) I would have actually done it like this:

def sum(ls: List[Int]) = ls.foldLeft(0) { _ + _ }

Also statically safe, but not as clear cut.

def sum(ls: List[Int]) = ls.foldLeft(0) { _ + _ }

                                        ^^^^^^^^^

Any language that accepts a moon smiley as valid code, has got to be good. :-)

I sort of like the implicit _ params in Scala. Pros and cons vs. the it of Fan, but overall (lacking tons of experience with either, admittedly), I like Fan's use of it better.

edit: deleted

both languages are statically typed alternatives on the JVM (unlike the dynamic languages such as Groovy....

Small correction: Groovy allows you to use static and/or dynamic typing.

Small correction: Groovy allows you to use static and/or dynamic typing.

I remember reading at some point that they don't generate optimized bytecode based on explicit variable types. And not because they didn't take to time to do that but because they ended up choosing semantics that didn't allow for it. In a quick skim of Groovy's docs on explicit types, I didn't see why they couldn't. Either my skim was too quick, the docs are long out of date, or my prior understanding was wrong. I'm not sure which of the three is the case.

>Small correction: Groovy allows you to use static and/or dynamic typing.

It's part static and part dynamic 8)

See Groovy Does Not Have Optional Static Typing

jfheon, thanks for the pointer.

jfheon

Small correction: Groovy allows you to use static and/or dynamic typing.

It's part static and part dynamic 8)

See Groovy Does Not Have Optional Static Typing

Point taken. Thanks!

@tompalmer:

When first seeing Scala (and having done Groovy in the past), I did like it better than _

But it fails. When reading

def sum(ls: List[Int]) = ls.foldLeft(0) { it + it }

the developer mind (at least mine) would assume it to be the same value / variable, but not _ in the case of

def sum(ls: List[Int]) = ls.foldLeft(0) { _ + _ }

because _ + _ is more like *.* where both * do not mean the same thing but are placeholders. It is not looking like a placeholder.

Cheers

Stephan

http://www.codemonkeyism.com

It should be called thingy instead of it :)

the developer mind (at least mine) would assume it to be the same value / variable

Understood, and in Fan it is the same variable. I had been referring to the whole package of semantics in each language on this matter, not just the syntax of it vs. _. My apologies on being unclear.

@djspiewak

def sum(ls: List[Int]) = ls.foldLeft(0) { _ + _ }

In this case, why wouldn't you use reduceLeft instead? I checked, and both work / give same answer...

scala> def sum(ls: List[Int]) = ls.foldLeft(0) { _ + _ }
sum: (List[Int])Int

scala> def sum2(ls: List[Int]) = ls.reduceLeft { _ + _ } 
sum2: (List[Int])Int

scala> var a = List(1,2,3,4,5) 
a: List[Int] = List(1, 2, 3, 4, 5)

scala> sum(a)
res3: Int = 15

scala> sum2(a)
res4: Int = 15

But IMO reduceLeft is conceptually easier because you don't specify the starting object.

@astubbs: You totally missed the point of djspiewak's first algorithms: The if-clause resp. the case Nil construct.

scala> def sum2(ls: List[Int]) = ls.reduceLeft { _ + _ }
sum2: (List[Int])Int

scala> val a = List[Int]()
a: List[Int] = List()

scala> sum2(a)
java.lang.UnsupportedOperationException: Nil.reduceLeft
        at scala.List.reduceLeft(List.scala:1093)
        at .sum2(<console>:4)
        at .<init>(<console>:7)
        at .<clinit>(<console>)
        at RequestResult$.<init>(<console>:3)
        at RequestResult$.<clinit>(<console>)
        at RequestResult$result(<console>)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorI...

Clear indeed: What should reduceLeft return on an empty list? For foldLeft this is simply the seed.

Thanks for taking on Scala's modules! In general, I expect that by dropping static types, Fan will be able to further the frontier in modules. For example, virtual classes look useful for modules, but are murderous to type.

That said, please be aware that the Scala supports modularity in ways far better than Java packages. Also, modules certainly have been considered in Scala's history. To be brief, the Scala approach to modules is to try and make the regular language features usable for large-scale components. A module instance is, in Scala, an object. If you have a module that is parameterizable -- for example, a sorting module parameterized by the ordering function and the element type -- then you'd make either a trait on an abstract class that takes those parts as parameters.

Maybe try a harder challenge? One problem considered in Scala's history was the "extensible expression problem". Maybe try that in Fan and see how it goes? The challenge is to design a module of expressions and operations on expressions. Then, define sub-modules that add new types of expressions, new operations, or both. OO and functional languages are good at one kind of extensibility or the other, but it's really tough to figure out a way to support both.

Lex,

Thanks for swinging by and giving some background on Scala.

Maybe try a harder challenge? One problem considered in Scala's history was the "extensible expression problem".

The way I understand this problem is how to you jointly extend your data classes and operations on those classes.

One aspect of the problem is that you have to do this with compile-time safety. I don't buy this aspect of the problem - large software systems aren't compiled, they are assembled (typically from pre-compiled modules). I don't personally think a compiler is the right tool for managing the assembly of modules. So for discussion sake, let's take the compile time safety requirement off the table (which probably makes the problem much less interesting if you care deeply about static type systems).

The way Fan treats this sort of extensibility is with its type database. Effectively, we redefine the problem, instead of a typing problem we think about it as database problem. All the pods (modules) installed contain types, and those types have relationships which are captured by facets (annotations). This can be indexed into a database which lets us query relationships between types. What types and relationships are available aren't known until deployment time, but it makes for amazingly extensible software systems.

An example in Fan:

class Data {}

class Operation 
{
  static Operation findFor(Data d) { Type.findByFacet(@opOnData, d.type, true) }
  abstract Void operate(Data d)
}

@opOnData=DataA# OperationA { ... }  class DataA {}
@opOnData=DataB# OperationB { ... }  class DataB {}

Operation.findFor(someData).operate(someData)

We are using reflection, casting, and maybe dynamic typing - so we haven't solved the problem with compile time safety. But I can drop in new pods which define either new Data classes or new Operation classes and they will be glued together at runtime. So things are quite extensible.

In my experience a strong static type system works really well "in the small" down at the low levels of a software architecture. But as you rise up to the top of an architecture and the problem becomes modeling data and gluing modules together, static typing becomes a straight jacket. I like Scala's consistent philosophy on the issue, but I definitely have a different philosophy.