I'm probably going to start working on the Fan JavaScript compiler next week. My plan right now is to emit JavaScript source code straight from the AST. And each pod will get a single JavaScript source file including every type. That seems the easiest thing to do, and we can see how well that works out.
I'm thinking we'll use facets to denote what API's are accessible from JavaScript, probably using @javascript:
@javascript
class Foo { ... } // all methods/fields accessible
class Bar
{
@javascript
Void a() { .. } // accessible to JS
Void b() { ... } // not accessible to JS
}
@javascript
class Foo
{
Void a() { ... } // accessible to JS
@javascript=false
Void b() { ... } // not accessible to JS
}
I'll most likely use Rhino for the JavaScript engine. After I get the sys pod flushed out, I'll nail down how this plugs into the DOM. I was originally going to have a Fan DOM stub API, but now I'm thinking I'm just going to use dynamic invoke for all that. We'll have to see though.
tompalmerThu 4 Dec 2008
Sounds great. To me, compilation to JavaScript seems vital in today's world. (And it's one of the important things haXe or even Java or C# has on Fan.)
I personally think fcode to JavaScript would be nicer than AST to JavaScript. I can imagine folks shipping pods without source, and it would be nice to have ability to convert those to JS, too.
As for the @javascript facet, I have mixed feelings. I can imagine wanting that control, but I can imagine it encouraging code not portable to other base platforms. But I can also imagine either alternative (exported or not) as default depending on the use case. Hmm.
andyThu 4 Dec 2008
As for the @javascript facet, I have mixed feelings. I can imagine wanting that control, but I can imagine it encouraging code not portable to other base platforms
That's a good point. And it has been bothering me that we're coupling the JavaScript runtime so tightly. So maybe its better to make it a blacklist - where everything is assumed to be compatible, but I explicitly make things inaccessible:
@javascript=false
class Foo { ... } // whole type not accessible to JS
class Foo
{
Void a() { ... } // accessible to JS
@javascript=false
Void b() { ... } // not accessible to JS
}
Of course the other option is just leave this off altogether - though there seems to be alot of value in being able to catch those errors in the compiler with a nice message, vs getting something like a not defined runtime error.
I'm open to suggestions, whats everyone else think?
tompalmerThu 4 Dec 2008
Maybe a general @export=false that applies to all platforms or some way to control exporting based on specified facets in build.fan? Just some thoughts.
jodastephenSat 6 Dec 2008
The @javascript concept seems quite intrusive to the main codebase. If you are just browsing the sourcebase while targetting Java do you want to see Javascript annotations?
The only alternative I can think of is a separate declarative file that lists the exclusions. The file could probably be a fog file. Then, IDEs could merge the external file with the main sourcebase to "hide" methods and classes if the user clicks javascript as a target.
If you do use annotations, they should be minimal, so defaulting to javascript true looks better.
Finally, I would prefer to see an fcode based system, as I think that level could become more important over time.
brianSat 6 Dec 2008
As for the @javascript facet, I have mixed feelings.
I don't think the core issue is necessarily the @javascript facet. It is more about how much we try to apply static typing to APIs used by JavaScript. We can assume any pod written 100% in Fan can be 100% compiled into JavaScript. So the issue is really only code which uses native methods and FFI calls. Pods like inet and fwt don't make sense to use in a browser environment at all. That really only leaves sys as the big special case. Something like sys is probably better annotated out-of-band.
We also have to consider documentation and IDE auto-completion. Does the main doc for a sys API show what is available in JavaScript? If not how to do I find that information in an easy way? This is sort of like .NET tries to solve with C#, C++, and VB examples. If the compiler knows what you can and cannot call we could just not document it at all - you'll get a compiler error (although that seems kind of lame).
I personally think fcode to JavaScript would be nicer than AST to JavaScript.
Fcode to JavaScript would definitely be nicer, but it is very difficult to disassemble fcode back to efficient imperative code. Some VMs like tamarin support execution of bytecode which would be nice to target, but unfortunately the universal deployment format is script text. The GWT team made the same decision to compile from source instead of bytecode (good discussion). I've actually written a Java bytecode to C compiler, and in the end I gave up on it because it was too hard to make the code efficient (that is how Sedona was born).
JohnDGMon 8 Dec 2008
I do not like the idea of facets. I would prefer the ability to compile any code to JavaScript.
There are three features I would really like to see:
It should be possible to call JavaScript libraries from Fan (with the help of Fan's already-strong dynamic features).
It should be possible to define JavaScript versions of "native" libraries; so, for example, one could implement a file library that saves data to a server using AJAX; or a browser-based version of FWT.
The serialization formats should be identical across Fan and JavaScript.
JohnDGMon 8 Dec 2008
Ooh, and one more: it should be possible to call Fan from JavaScript (this implies AST to JS).
brianMon 8 Dec 2008
I do not like the idea of facets. I would prefer the ability to compile any code to JavaScript.
That is definitely the intent. And yes we want to define JavaScript versions of "native" libraries. The issue is that we won't actually define a mapping for every native library (at least that I can envision). So if there is no mapping for File.mmap and you try to call it JavaScript do you get a compiler error or a runtime error? That was the question being debated I believe.
Ooh, and one more: it should be possible to call Fan from JavaScript (this implies AST to JS).
I'm not sure that requirement must be met by the JS generated from AST. That requirement just means that public Fan APIs map to "clean" JavaScript APIs (it shouldn't really matter how the code looks inside those method bodies).
Andy will definitely need to draft up some use cases to help solidify all these concepts.
JohnDGTue 9 Dec 2008
So if there is no mapping for File.mmap and you try to call it JavaScript do you get a compiler error or a runtime error? That was the question being debated I believe.
A runtime error. It's not decidable if a given snippet of JavaScript actually implements a function, so the only way to make it a compile error is by adding some form of annotations, which will pollute the purity of the code, duplicate content inside the JavaScript implementation, and diverge from that implementation over time.
I'm not sure that requirement must be met by the JS generated from AST.
You do not lose any high-level information going from AST to fcode?
An fcode implementation would be much slower, though I like the fact it would "obfuscate" internal method bodies, whilst still enabling the methods to be called from JavaScript.
brianWed 10 Dec 2008
You do not lose any high-level information going from AST to fcode?
I don't think you loose any high level information because pretty much everything available in a public API is available via reflection from the meta-data in the pod. So it is really more a question of the code itself.
We are definitely planning on mapping from AST not fcode, but just so everyone understands the difference of what the output JavaScript would look like:
Fan code:
Int add(Int a, Int b) { return a + b }
From AST we map statements and expressions:
function add(a, b) { return a + b }
From fcode we map stack operations and typically use local variables to represent the "stack":
function add(a, b)
{
stack1 = a
stack2 = b
stack1 = stack1 + stack2
return stack
}
geme_hendrixFri 3 Jul 2009
Just to clarify, are we going to have the same sort of native method feature set in Fan like GWT has?
I see lots of use cases to conveniently map existing powerful JavaScript libraries and hence wrap them up in Fan - which would then obviously then get compiled back down to JavaScript :¬).
andyFri 3 Jul 2009
I haven't used GWT so not sure what exactly it has. But we do currently support the native keyword lets you dive into Javascript. This is how the FWT has been implemented, as well as Fan wrappers for things like Google Maps. So it I think the answers is yes?
andy Thu 4 Dec 2008
I'm probably going to start working on the Fan JavaScript compiler next week. My plan right now is to emit JavaScript source code straight from the AST. And each pod will get a single JavaScript source file including every type. That seems the easiest thing to do, and we can see how well that works out.
I'm thinking we'll use facets to denote what API's are accessible from JavaScript, probably using
@javascript
:I'll most likely use Rhino for the JavaScript engine. After I get the sys pod flushed out, I'll nail down how this plugs into the DOM. I was originally going to have a Fan DOM stub API, but now I'm thinking I'm just going to use dynamic invoke for all that. We'll have to see though.
tompalmer Thu 4 Dec 2008
Sounds great. To me, compilation to JavaScript seems vital in today's world. (And it's one of the important things haXe or even Java or C# has on Fan.)
I personally think fcode to JavaScript would be nicer than AST to JavaScript. I can imagine folks shipping pods without source, and it would be nice to have ability to convert those to JS, too.
As for the
@javascript
facet, I have mixed feelings. I can imagine wanting that control, but I can imagine it encouraging code not portable to other base platforms. But I can also imagine either alternative (exported or not) as default depending on the use case. Hmm.andy Thu 4 Dec 2008
That's a good point. And it has been bothering me that we're coupling the JavaScript runtime so tightly. So maybe its better to make it a blacklist - where everything is assumed to be compatible, but I explicitly make things inaccessible:
Of course the other option is just leave this off altogether - though there seems to be alot of value in being able to catch those errors in the compiler with a nice message, vs getting something like a
not defined
runtime error.I'm open to suggestions, whats everyone else think?
tompalmer Thu 4 Dec 2008
Maybe a general
@export=false
that applies to all platforms or some way to control exporting based on specified facets inbuild.fan
? Just some thoughts.jodastephen Sat 6 Dec 2008
The
@javascript
concept seems quite intrusive to the main codebase. If you are just browsing the sourcebase while targetting Java do you want to see Javascript annotations?The only alternative I can think of is a separate declarative file that lists the exclusions. The file could probably be a fog file. Then, IDEs could merge the external file with the main sourcebase to "hide" methods and classes if the user clicks javascript as a target.
If you do use annotations, they should be minimal, so defaulting to javascript true looks better.
Finally, I would prefer to see an fcode based system, as I think that level could become more important over time.
brian Sat 6 Dec 2008
I don't think the core issue is necessarily the @javascript facet. It is more about how much we try to apply static typing to APIs used by JavaScript. We can assume any pod written 100% in Fan can be 100% compiled into JavaScript. So the issue is really only code which uses native methods and FFI calls. Pods like
inet
andfwt
don't make sense to use in a browser environment at all. That really only leavessys
as the big special case. Something likesys
is probably better annotated out-of-band.We also have to consider documentation and IDE auto-completion. Does the main doc for a sys API show what is available in JavaScript? If not how to do I find that information in an easy way? This is sort of like .NET tries to solve with C#, C++, and VB examples. If the compiler knows what you can and cannot call we could just not document it at all - you'll get a compiler error (although that seems kind of lame).
Fcode to JavaScript would definitely be nicer, but it is very difficult to disassemble fcode back to efficient imperative code. Some VMs like tamarin support execution of bytecode which would be nice to target, but unfortunately the universal deployment format is script text. The GWT team made the same decision to compile from source instead of bytecode (good discussion). I've actually written a Java bytecode to C compiler, and in the end I gave up on it because it was too hard to make the code efficient (that is how Sedona was born).
JohnDG Mon 8 Dec 2008
I do not like the idea of facets. I would prefer the ability to compile any code to JavaScript.
There are three features I would really like to see:
JohnDG Mon 8 Dec 2008
Ooh, and one more: it should be possible to call Fan from JavaScript (this implies AST to JS).
brian Mon 8 Dec 2008
That is definitely the intent. And yes we want to define JavaScript versions of "native" libraries. The issue is that we won't actually define a mapping for every native library (at least that I can envision). So if there is no mapping for
File.mmap
and you try to call it JavaScript do you get a compiler error or a runtime error? That was the question being debated I believe.I'm not sure that requirement must be met by the JS generated from AST. That requirement just means that public Fan APIs map to "clean" JavaScript APIs (it shouldn't really matter how the code looks inside those method bodies).
Andy will definitely need to draft up some use cases to help solidify all these concepts.
JohnDG Tue 9 Dec 2008
A runtime error. It's not decidable if a given snippet of JavaScript actually implements a function, so the only way to make it a compile error is by adding some form of annotations, which will pollute the purity of the code, duplicate content inside the JavaScript implementation, and diverge from that implementation over time.
You do not lose any high-level information going from AST to fcode?
An fcode implementation would be much slower, though I like the fact it would "obfuscate" internal method bodies, whilst still enabling the methods to be called from JavaScript.
brian Wed 10 Dec 2008
I don't think you loose any high level information because pretty much everything available in a public API is available via reflection from the meta-data in the pod. So it is really more a question of the code itself.
We are definitely planning on mapping from AST not fcode, but just so everyone understands the difference of what the output JavaScript would look like:
Fan code:
From AST we map statements and expressions:
From fcode we map stack operations and typically use local variables to represent the "stack":
geme_hendrix Fri 3 Jul 2009
Just to clarify, are we going to have the same sort of native method feature set in Fan like GWT has?
I see lots of use cases to conveniently map existing powerful JavaScript libraries and hence wrap them up in Fan - which would then obviously then get compiled back down to JavaScript :¬).
andy Fri 3 Jul 2009
I haven't used GWT so not sure what exactly it has. But we do currently support the
native
keyword lets you dive into Javascript. This is how the FWT has been implemented, as well as Fan wrappers for things like Google Maps. So it I think the answers is yes?