I am not sure how many people in the wild are writing Fan code, but I know we are starting to write a lot of production code in Fan. Over the course of the next six months, I will probably also have many partners starting to write Fan code too. So the window for introducing breaking changes is fast coming to a close.
I think we are still a ways away from declaring a locked down 1.0 release, but I am going to start setting the bar pretty high for introducing new language features. We need to have a period of stability which is critical for two reasons. First people need to have a stable platform to start using Fan. Second we need to get experience under our belt with Fan's existing feature set. The more features you add, the more you limit yourself in the future - yet you tend to have a much better perspective in the future.
With that said, I would like to open discussions about what people feel we must solve before locking down the language for a 1.0 release. If you feel something must be solved, now is the time to get it on the table.
For now I will consider the closure and with-block discussion decided per previous discussion (although maybe it can never die).
Perhaps the biggest outstanding issue is Fan's open constructors where any client can set public fields via a with-block. I see an issue there, but at the same time this isn't something that really bothers me. It can be solved using private fields which are controlled by constructors, but I admit that solution is a bit lacking.
There is still a hole in the nullable type system in that I do not check if all constructor paths have assigned a value to a non-nullable field. The issue with "fixing" this is that I have all sorts of code which make this a big pain in the ass: cases where I don't want to assign a value until later in the object's life cycle, but I don't ever want to assign null. So I like the current design, but also realize it is a bit ugly - so discussions welcome.
I think a nice addition would be remove some of the boiler plate code for "struct classes" which require passing values into the constructor, and implementing equals and hash. My fear here is that an early solution in 1.0 will likely conflict with more powerful solutions in the future such as tuples or anonymous classes.
brianSun 25 Jan 2009
One more thought: remember that the DSL feature is our escape hatch. It lets us plug language holes and experiment with new language features without risking the core language. This the category where I lump things like list comprehensions, pattern matching, LINQ, etc. Let's allow these features to play out in DSLs in the community before committing them to the core language.
JohnDGSun 25 Jan 2009
Clean, well-defined distinction between with-block construction (with post validation) and ordinary with-block usage.
Working concurrency. I personally don't believe that messaging is a good solution for the concurrency problem, because it's not easy to program using the model and it doesn't truly solve concurrency issues -- it just makes it harder to get to the point where you see them (even if you don't include it in Fan, you will see third-party libs that will block until they receive a particular message, which is the first step towards deadlocks). Nonetheless, Fan has adopted the messaging model and needs to develop it to a point where it's useful and easy to use.
DSLs. I think it's important this feature be in 1.0 because it will impact the development of third-party libraries. That is, if library writers have access to DSLs, they will produce dramatically different libraries than if they do not have access to DSLs. You don't want to piss off the early adopters by forcing them down dead end paths.
Standardized, well-defined infrastructure and architecture for cross-compilation. That is, if someone ones to write a JavaScript/Java/.NET version of some library, they should take the same steps, regardless of the target language (maybe this is already true).
Remove the magic from List and Map. These have extra help from the language, which a third-party collections library will not have. Consequently, it biases libraries to use these structures even though some of us would prefer to use other collection APIs.
There are a few warts and inconsistencies in the language, but in my opinion, nothing that should prevent a 1.0 release.
One thing I would be strongly in favor of, which I don't include in the above list, is null safety (that is, it should not be possible to assign a non-nullable instance to a nullable instance, unless there is a conditional that verifies the instance is not null). This is the kind of thing that if it's not in the 1.0 release, it's never going to be there, because no one's going to want to rewrite all the code that would be affected by such a change (for this kind of a change, it's not possible to automatically translate the code because you have to decide what to do if something might be null). Concurrently, the standard libraries should be tweaked to minimize use of null as much as possible (I've given examples before).
cases where I don't want to assign a value until later in the object's life cycle, but I don't ever want to assign null.
This is often an anti-pattern that occurs because an object has acquired more than one concern. A rule that I follow: objects should always exist in valid states and should not be able to transition into invalid states. This rule leads to a few more objects, but a better design.
In cases where the above is not an anti-pattern, it's usually possible to refactor the code to use a builder object, which has setters and a build method, but no way to access intermediate values. I imagine it would be possible to create a generic Builder using Fan's dynamics.
brianSun 25 Jan 2009
Clean, well-defined distinction between with-block construction (with post validation) and ordinary with-block usage.
Not sure we will ever reach an elegant solution with good performance given how much discussion we've already had on the matter. But something to decide one way or the other.
Working concurrency. I personally don't believe that messaging is a good solution for the concurrency problem
I've been creating some fairly complex things with the existing Fan messaging model. Except for a couple things, I think it works pretty well. It is a zillion times better than Java's concurrency model. You are right that sendSync can deadlock, but a really simple pattern (or maybe even a hard rule) is not to allow anyone who accepts messages to use sendSync (only client threads can use it, not service threads). But this is an area of concern which will never really be finished. Given that I'm in the process of implementing a highly distributed real-time database, I think the concurrency stuff will get hammered pretty hard.
DSLs. I think it's important this feature be in 1.0
My plan is to have DSLs prototyped for the next build (or at least in the next month).
Standardized, well-defined infrastructure and architecture for cross-compilation.
I think this is mostly in place today - you can compile using the Java or the .NET runtime. It is the FFI where trouble arises - getting the Java FFI to compile on .NET doesn't work today, but isn't too big a deal. But I expect getting the .NET FFI to compile on Java runtime is going to suck.
Remove the magic from List and Map. Consequently, it biases libraries to use these structures even though some of us would prefer to use other collection APIs
This is by design - List and Map are special just like Bool, Int, and Str are special - these are the normalized representations of key types which should be used as the public interfaces of your APIs. Not that we shouldn't allow other collection types, but I view those as collections for implementing your APIs, not exposing in your API.
that is, it should not be possible to assign a non-nullable instance to a nullable instance, unless there is a conditional that verifies the instance is not null
I think we have hashed this out before. It makes sense in a more strongly typed language, but not in Fan where we implicitly cast most things.
This is often an anti-pattern that occurs because an object has acquired more than one concern.
I agree it tends be a bit ugly, but it does happen. Consider the compiler's AST. Each Expr has a field for the expression's type which starts life as null until further in the compiler pipeline when we can compute the type. At no time should we ever explicitly assign null, but then again there is no good value but null to initialize it with. You could certainly work around to create some type of NotInitializedErr, but I'm not sure that is much better than NullErr. Probably a topic of worthy of its own debate.
JohnDGSun 25 Jan 2009
I agree it tends be a bit ugly, but it does happen. Consider the compiler's AST. Each Expr has a field for the expression's type which starts life as null until further in the compiler pipeline when we can compute the type.
In this case, Expr has acquired two concerns: an untyped expression and a typed expression. In an OOP language I would view TypedExpr as a subclass of Expr which extends the notion of an expression to include a type. Any code that operates on Expr will also operate on TypedExpr, but any code that requires a TypedExpr will be forbidden by the type system to operate on Expr.
Without generics, I don't see an easy way to thread typing through the tree. But you do use the visitor pattern, which could be used to reduce the need for casting a Node (possibly incorrectly) into a TypedExpr when it's really just an Expr.
jodastephenMon 26 Jan 2009
This is partly a question about what can be added later if we deem it necessary, and whether you will have breaking changes in a v2.0 of Fan. It might be worth reserving some keywords now even though you don't use them (do, goto, checked, unchecked)
While I do have some issues at the language level, some key ones have been resolved (non-null, DSLs, closures). Here are some others with priority (which I've mostly mentioned before):
construction - Critical. Objects need to be verified in a uniform way through the construction process, whether by deserialization, make, constructor or declarative).
immutable setters - Important. There needs to be a way to set the value of immutable fields (which returns a new instance of the parent)
value objects - Important. Quick and easy way to define value classes with low ceremony constructors, equals, hashcode and optional compare.
resource access - Important. There must be a way to cleanup after resources, like C# using, or Java ARM. It might be worth prototyping this as a DSL, even if its just to check that model out.
composition - Important. Providing a way to easily wrap one class with another, like this but more.
facet namespacing - Significant. Namespacing would make the use of facets much more reliable than simply hoping that no one else has used the name you're using.
static contracts - Significant. Static contracts are the completion of the integration of static code with OO code. This is an edge case feature, but one that makes the language conceptually much more complete.
once methods on const classes - Annoying. Sometimes you want to cache the result of a hash code or to string calculation on a const class. This is safe, as the result of the calculation is always consistent.
duration literals - Annoying. They are too specific to be a direct literal. It would be fine if they were a specific example of a pluggable literal. (Plus as Mr date and time I have a problem with the days literal being 24 hours)
pod names - Annoying. comAcmeFooPod is a horrible convention, and one that companies won't like (as they want to have different security access rights to pods but can't because they are all in one directory)
map type syntax - Annoying. Feedback from developers in my company is that they don't like the two alternate forms of Map type syntax - \[Str:Int] and Str:Int. Personally, I don't like that these look nothing like the type for lists.
public scope by default - Annoying. Maybe controversial, but I believe that internal scope would be more appropriate as the default. That way, there has to be a positive decision to expose something outside the pod.
switch case statements curlies - Annoying. The case statements in switch should be allowed to have curly brace scopes.
Ideally each of these should be addressed, however the bear minimum is to address the Critical/Important ones (if we can clearly say that we can add it in 1.1 in a non-breaking way, then it could be deferred).
brianMon 26 Jan 2009
This is partly a question about what can be added later if we deem it necessary, and whether you will have breaking changes in a v2.0 of Fan.
I will consider breaking changes in the future, most likely with the ability to annotate a given source file as using an older version. But I guess we just need to play things by ear.
Ideally each of these should be addressed, however the bear minimum is to address the Critical/Important ones
There is a lot of stuff in that list, some of which we have already talked about. Maybe instead of trying to tackle it all at once, I would suggest opening up new posts for the issues you feel are critical to begin a discussion.
heliumMon 26 Jan 2009
I have a problem with the days literal being 24 hours
What's the problem with that?
once methods on const classes
Should definitely work.
tompalmerMon 26 Jan 2009
(Reformatted here.) My biggest breaking change recommendations:
Namespaced facets, as previously discussed.
Require commas following (or something to mark) the implicit use of add in with blocks.
My biggest non-breaking change recommendations:
Closure parameter type inference.
Single-expression auto-return closures.
Support for arbitrary co and contravariance on parameters (and return types) when overriding methods. (Behind the scenes, the parameter types would still match.) It would go hand in hand with the autocasting support and lack of generics.
JohnDGMon 26 Jan 2009
Support for arbitrary co and contravariance on parameters (and return types) when overriding methods.
I do agree this makes a lot of sense. I'd like to be able to override a method and supply any fitting return value or parameters. Since Fan doesn't allow method name overloading, it seems like this would be a pretty simple change, and would greatly improve API design.
For example, in an AST, it would allow you a typed version, which overloads various key methods to return TypedExpr instead of Expr. This is an alternate approach to generics, which Fan does not yet support.
Require commas following (or something to mark) the implicit use of add in with blocks.
I do like that.
immutable setters
For sure, they are important to avoid boilerplate. Might want to consider reserving the with keyword.
duration literals
I don't like the fact that it's impossible to create your own. Of course we can abuse DSLs to get the same effect. :-)
mass := Mass <| 14kg |>
brianMon 26 Jan 2009
I have posted two issues which seem to top both John and Stephen's lists and which I personally want to nail down:
I still have the same position as before - I would welcome a solution, but I am skeptical of any design which introduces new complexity (such as symbols). But if we want a solution, it is now or never - so if you care deeply please launch a new discussion topic with a proposal.
Require commas following (or something to mark) the implicit use of add in with blocks.
Andy and I actually talked about this last week. We went thru some of the pros and cons. There isn't actually a lot of ambiguity with the current design since I don't let you use just a simple local variable. Given the low probability of an issue versus the added syntactic noise (and the weird trailing comma for one item) we decided to stick with the current design. Although if anyone feels strongly, then please open a new discussion.
Tom I think your other issues can all be handled in a non-breaking way, so I think we can defer them.
brian Sun 25 Jan 2009
I am not sure how many people in the wild are writing Fan code, but I know we are starting to write a lot of production code in Fan. Over the course of the next six months, I will probably also have many partners starting to write Fan code too. So the window for introducing breaking changes is fast coming to a close.
I think we are still a ways away from declaring a locked down 1.0 release, but I am going to start setting the bar pretty high for introducing new language features. We need to have a period of stability which is critical for two reasons. First people need to have a stable platform to start using Fan. Second we need to get experience under our belt with Fan's existing feature set. The more features you add, the more you limit yourself in the future - yet you tend to have a much better perspective in the future.
With that said, I would like to open discussions about what people feel we must solve before locking down the language for a 1.0 release. If you feel something must be solved, now is the time to get it on the table.
For now I will consider the closure and with-block discussion decided per previous discussion (although maybe it can never die).
Perhaps the biggest outstanding issue is Fan's open constructors where any client can set public fields via a with-block. I see an issue there, but at the same time this isn't something that really bothers me. It can be solved using private fields which are controlled by constructors, but I admit that solution is a bit lacking.
There is still a hole in the nullable type system in that I do not check if all constructor paths have assigned a value to a non-nullable field. The issue with "fixing" this is that I have all sorts of code which make this a big pain in the ass: cases where I don't want to assign a value until later in the object's life cycle, but I don't ever want to assign null. So I like the current design, but also realize it is a bit ugly - so discussions welcome.
I think a nice addition would be remove some of the boiler plate code for "struct classes" which require passing values into the constructor, and implementing equals and hash. My fear here is that an early solution in 1.0 will likely conflict with more powerful solutions in the future such as tuples or anonymous classes.
brian Sun 25 Jan 2009
One more thought: remember that the DSL feature is our escape hatch. It lets us plug language holes and experiment with new language features without risking the core language. This the category where I lump things like list comprehensions, pattern matching, LINQ, etc. Let's allow these features to play out in DSLs in the community before committing them to the core language.
JohnDG Sun 25 Jan 2009
List
andMap
. These have extra help from the language, which a third-party collections library will not have. Consequently, it biases libraries to use these structures even though some of us would prefer to use other collection APIs.There are a few warts and inconsistencies in the language, but in my opinion, nothing that should prevent a 1.0 release.
One thing I would be strongly in favor of, which I don't include in the above list, is null safety (that is, it should not be possible to assign a non-nullable instance to a nullable instance, unless there is a conditional that verifies the instance is not null). This is the kind of thing that if it's not in the 1.0 release, it's never going to be there, because no one's going to want to rewrite all the code that would be affected by such a change (for this kind of a change, it's not possible to automatically translate the code because you have to decide what to do if something might be null). Concurrently, the standard libraries should be tweaked to minimize use of null as much as possible (I've given examples before).
You might be interested in The Billion Dollar Mistake.
This is often an anti-pattern that occurs because an object has acquired more than one concern. A rule that I follow: objects should always exist in valid states and should not be able to transition into invalid states. This rule leads to a few more objects, but a better design.
In cases where the above is not an anti-pattern, it's usually possible to refactor the code to use a builder object, which has setters and a build method, but no way to access intermediate values. I imagine it would be possible to create a generic
Builder
using Fan's dynamics.brian Sun 25 Jan 2009
Not sure we will ever reach an elegant solution with good performance given how much discussion we've already had on the matter. But something to decide one way or the other.
I've been creating some fairly complex things with the existing Fan messaging model. Except for a couple things, I think it works pretty well. It is a zillion times better than Java's concurrency model. You are right that
sendSync
can deadlock, but a really simple pattern (or maybe even a hard rule) is not to allow anyone who accepts messages to usesendSync
(only client threads can use it, not service threads). But this is an area of concern which will never really be finished. Given that I'm in the process of implementing a highly distributed real-time database, I think the concurrency stuff will get hammered pretty hard.My plan is to have DSLs prototyped for the next build (or at least in the next month).
I think this is mostly in place today - you can compile using the Java or the .NET runtime. It is the FFI where trouble arises - getting the Java FFI to compile on .NET doesn't work today, but isn't too big a deal. But I expect getting the .NET FFI to compile on Java runtime is going to suck.
This is by design - List and Map are special just like Bool, Int, and Str are special - these are the normalized representations of key types which should be used as the public interfaces of your APIs. Not that we shouldn't allow other collection types, but I view those as collections for implementing your APIs, not exposing in your API.
I think we have hashed this out before. It makes sense in a more strongly typed language, but not in Fan where we implicitly cast most things.
I agree it tends be a bit ugly, but it does happen. Consider the compiler's AST. Each Expr has a field for the expression's type which starts life as null until further in the compiler pipeline when we can compute the type. At no time should we ever explicitly assign null, but then again there is no good value but null to initialize it with. You could certainly work around to create some type of
NotInitializedErr
, but I'm not sure that is much better thanNullErr
. Probably a topic of worthy of its own debate.JohnDG Sun 25 Jan 2009
In this case,
Expr
has acquired two concerns: an untyped expression and a typed expression. In an OOP language I would viewTypedExpr
as a subclass ofExpr
which extends the notion of an expression to include a type. Any code that operates onExpr
will also operate onTypedExpr
, but any code that requires aTypedExpr
will be forbidden by the type system to operate onExpr
.Without generics, I don't see an easy way to thread typing through the tree. But you do use the visitor pattern, which could be used to reduce the need for casting a
Node
(possibly incorrectly) into aTypedExpr
when it's really just anExpr
.jodastephen Mon 26 Jan 2009
This is partly a question about what can be added later if we deem it necessary, and whether you will have breaking changes in a v2.0 of Fan. It might be worth reserving some keywords now even though you don't use them (do, goto, checked, unchecked)
While I do have some issues at the language level, some key ones have been resolved (non-null, DSLs, closures). Here are some others with priority (which I've mostly mentioned before):
Ideally each of these should be addressed, however the bear minimum is to address the Critical/Important ones (if we can clearly say that we can add it in 1.1 in a non-breaking way, then it could be deferred).
brian Mon 26 Jan 2009
I will consider breaking changes in the future, most likely with the ability to annotate a given source file as using an older version. But I guess we just need to play things by ear.
There is a lot of stuff in that list, some of which we have already talked about. Maybe instead of trying to tackle it all at once, I would suggest opening up new posts for the issues you feel are critical to begin a discussion.
helium Mon 26 Jan 2009
What's the problem with that?
Should definitely work.
tompalmer Mon 26 Jan 2009
(Reformatted here.) My biggest breaking change recommendations:
add
in with blocks.My biggest non-breaking change recommendations:
JohnDG Mon 26 Jan 2009
I do agree this makes a lot of sense. I'd like to be able to override a method and supply any fitting return value or parameters. Since Fan doesn't allow method name overloading, it seems like this would be a pretty simple change, and would greatly improve API design.
For example, in an AST, it would allow you a typed version, which overloads various key methods to return
TypedExpr
instead ofExpr
. This is an alternate approach to generics, which Fan does not yet support.I do like that.
For sure, they are important to avoid boilerplate. Might want to consider reserving the
with
keyword.I don't like the fact that it's impossible to create your own. Of course we can abuse DSLs to get the same effect. :-)
brian Mon 26 Jan 2009
I have posted two issues which seem to top both John and Stephen's lists and which I personally want to nail down:
Regarding Tom's key issues:
I still have the same position as before - I would welcome a solution, but I am skeptical of any design which introduces new complexity (such as symbols). But if we want a solution, it is now or never - so if you care deeply please launch a new discussion topic with a proposal.
Andy and I actually talked about this last week. We went thru some of the pros and cons. There isn't actually a lot of ambiguity with the current design since I don't let you use just a simple local variable. Given the low probability of an issue versus the added syntactic noise (and the weird trailing comma for one item) we decided to stick with the current design. Although if anyone feels strongly, then please open a new discussion.
Tom I think your other issues can all be handled in a non-breaking way, so I think we can defer them.