The next major feature I will be tackling is interop with Java and .NET libraries. The work during the past four weeks has all been to lay the foundation for interop in the Java and .NET runtimes. Now it is time to tackle the language and compiler side of the problem.

The design I'm thinking about it is a way to allow compiler plugins which deal with an external "namespace". External namespace types would be brought into the Fan type as normal classes and mixins, then used just like normal Fan types. Key differences would be that external namespaces might:

• might require a thunk to translate things like primitives, arrays, etc
• might allow method overloading by parameters

Handling issues like the above when calling out to external types is fairly straight forward. However letting a Fan class subclass/implement an external type has some tricky complications.

Syntax

One of the first things to decide is an enhanced using statement syntax which specifies two things: the name of the compiler plugin being used and a namespace definition meaningful to that plugin. I'm thinking about something like this:

// generic
using plugin:namespace
using plugin:namespaceType
using plugin:namespaceType as rename

// java
using java:javax.swing               // import javax.swing.*
using java:java.util.Date            // import just java.util.Date
using java:java.util.Date as JDate   // import Date as JDate

// .NET
using dotnet:System.Date
using dotnet:System.Data.DataSet

I'm not fond of the colon syntax, especially since Fan qualified names use double colons. Other suggestions are welcome.

Compiler Plugin

When the Fan compiler sees an external using statement, it will look in the type database to find the plugin registered for that name. The plugin will then be responsible for:

• Importing the the types of the external namespace into that compilation unit's symbol tables
• Resolving slots against the external types (fields, methods)
• Type checking field access and method calls
• Generating assembly of external calls

Once we've established a comfort level with the design, phase 2 would be to try and allow Fan classes to extend external types (for example allow Fan to extend a Java class or implement an interface). This will involve more advanced plugin functionality:

• Allowing plugin to participate in the Inherit step
• Figure out how Fan overrides methods with primitives/arrays
• Figure out how Fan overrides methods overloaded by parameters

Runtime Enhancements

My thinking is that the output of Fan code which uses external types is still normal fcode. The plugin will get a chance to emit normal Fan field access and method calls when it makes sense. For special cases, we'll add a new opcode which will allow the runtimes to emit a thunk to handle primitives, arrays, etc.

Another aspect of the runtime to consider is how an external type maps the reflection APIs and handles dynamic invoke. My current thinking is that a Java/.NET type wouldn't return anything for the reflection APIs - either throw exception or just return sys::Obj slots. However we would override trap so that you could use dynamic calls against Java/.NET types.

Java Plugin

I'll be prototyping this effort for Java first. Then Andy will be doing the .NET side once we've figured it out. To keep things simple I'm going to use normal Java reflection to query for Java classes. This will let me reuse the same code for both the compiler side and the dynamic invoke runtime side. That means anything you want to compile against will need to be in the normal classpath.

So those are some rough thoughts to the design I'm considering. Obviously there are a ton of issues to work through - I'm just trying to lay the basic concepts here.

.Net will be a lot harder than Java as .Net has a lot of things Java dosn't.

• ref and out parameters.
• Generics without type erasure
• delegates
• events
• ...

For the first round, I recommend making anything uncertain illegal. As in, you can't override overloaded methods. Better could be done, but I think it would be an okay starting point.

I hope you could move away from reflection in the compiler eventually, but it's an easy start. In my own toy efforts before, I was using ASM to avoid such issues, but even still I cheated some.

Are you still planning to make it easy to have different files with implementations for different underlying platforms? I don't think it should be required, say if I only care about Java, and the ability to write the different bindings directly in Fan would be sweet, but still the ability to easily make different implementations for those who care would be nice.

Also, I recommend runtime errors (such as CastErr perhaps?) for cases of trying to stuff a Fan Int into say a Java byte if the value is too large or small. So, it would require bounds checking at every stuff point. But that's my recommendation.

Here, byte would represent a secret subtype of Int that you happen to match if your Int is small enough. Maybe would work.

.Net will be a lot harder than Java as .Net has a lot of things Java dosn't.

That is a good point. Many of those issues can be handled when calling out using normal methods and call opcodes. Generics will be tricky for both platforms. It is subclassing which is going to be the real beast.

For the first round, I recommend making anything uncertain illegal.

Yes I agree.

I hope you could move away from reflection in the compiler eventually

Yeap - I just think reflection is an easy way to get started. Of course we could use something like ASM, but ideally we want the disassembler be in Fan so that we can cross-compile for both Java and .NET.

So, it would require bounds checking at every stuff point.

I'm not so sure about that - seems like it could be expensive and on Java you typically require trunction when working with bytes (having bytes be signed is so freaking painful in Java). .NET actually has much better support for overflow.

Here's a suggestion for the using syntax that avoids the double colon preceding import name...

// generic
using(plugin) namespace
// java
using(java) javax.swing               
// .NET
using(dotnet) System.Date

or maybe this...

// generic
using:plugin namespace
// java
using:java javax.swing               
// .NET
using:dotnet System.Date

Actually I like both of those more than my original proposal. Note sure I have a strong preference either way. Any more votes on that syntax?

I like using:plugin namespace better. It is harder to mistype.

I like using(plugin) better. Just personal taste.

I don't have a strong opinion either, but I think using:plugin reads a bit better.

Is it necessary to say anything other than just plain using? It wouldn't make sense to mix Java and .NET in the same file, anyway. Couldn't the specifier of context be something other than the using statement?

Is the concern that Fan namespaces might collide with other systems? Or is it a parser issue since things like "com.whatever.something" don't exist in normal Fan? Or do we just like being explicit?

Is it necessary to say anything other than just plain using

I would prefer to have the using statement (or some other mechanism) explicitly define which plugin to use. The alternative would be to load all the plugins all the time, and query each one if they were responsible for a given namespace (and hope there were no conflicts). In this case I prefer the explicit design - it is only a few extra chars in your using statements, and seems a lot cleaner.

I tried out some various formats for FFI using statements. The using:java syntax doesn't work that great. My leading syntax right now is:

using [java] java.lang
using [java] java.util::Date as JavaDate

I haven't commented on this thread until now, as it fills me with a little unease at the number of complexities. For example, writing an IDE is going to get a lot harder. Maybe thats just an inevitable cost.

I can't say that the latest syntax proposal looks pretty either, but thats a taste thing. using:java looked more readable. Another option:

using java {
  java.lang
  java.util::Date as JavaDate
}

I'll start a new thread for my use case.

using:java looked more readable.

The problem is that 90% of the time you're importing a package which starts with java:

using:java java.lang
using:java java.util

I don't like combining the using keyword with a colon, and I want the two java's to stand out more:

using [java] java.lang
using [java] java.util

The other case using braces was something I tried out. But I prefer the using statements to just line up on single lines.

The using [java] java.util.* style is growing on me, I could live with it.

Actually the [java]java.lang format is working out pretty well. Basically what I'm doing is keeping all the same fcode infrastructure exactly as is and just generating a different format for the pod name. This format lets the compiler and runtime code efficiently check for a FFI method by just the checking if the first char is "[".