Environments are a very powerful extension of the Fantom language. I think they have many potential uses and I'd like to see them well supported by the rest of the language.
I want to propose an alternative to ticket Uri Namespace (#934). I believe many of the problems that proposal deals with can be resolved by clever use of environments. Arguably, of course, in a cleaner, more modular way.
The gist of this alternative proposal is:
to keep pod::Type.slot syntax for naming Fantom entities, and
to make sys::Env solely responsible for resolving simple pod names to actual pods.
Background
The problems this proposal and ticket #934 intend to solve are both fairly involved and leave a lot of room for debate. Rather than bog down the #934 thread, I wanted to fork it to propose a possible alternative here.
To summarize, #934 proposes three separate changes:
to remove the sys::UriSpace class
to overhaul the reflrection API to use URIs exclusively, and
to allow using statements to use URIs
I don't think anyone really misses sys::UriSpace all that much, so it will not be mentioned again.
As I've previous stated in the #934 thread, I don't like the URI syntax proposed for describing the pods. The pod::type.slot syntax is so impossibly simple. I feel it is the Right Thing to do.
My main argument is that the current environment should be the one and only part of the system which cares about where pod files are loaded from.
Using as "what" not "where"
What is the meaning of the using statement?
Currently, the using statement tells you "what" pod dependency exists for a Fantom source file. If you want to use any class from a pod, you must declare it as a dependency up at the top of your file.
Ticket #934 plans to change that meaning. According to the proposed changes, the using statement tells you "where" a dependency exists. You must specify the URI pointing to the pod you want.
According to my new proposal, the using statement remains a "what". The "where" of a dependency belongs exclusively to the current Env class. The Env class is also promoted in importance across the language.
Pod URIs and Envs are at odds
The problem with naming pods with URIs is that URIs often contain information about "where" a resource exists. With that comes assumptions about how a resource is to be resolved. For example, take a class named fan:[file:../shared/script.fan]::SomeClass (according to #934's URI conventions). There is a strong assumption that SomeClass's pod will be loaded from a file named script.fan. But ultimately, the decision of "where" a pod is loaded from rests with the current environment.
By allowing the location of a pod to be specified in two places, you create some bad redundancy. Most likely, the URI will end up "winning" and the usefulness of the Env mechanism is substantially diminished.
Your class doesn't really care "where" a pod comes from anyway. It just wants it to be there when the time comes. The environment is the only one who cares about "where", and it should be granted full authority on how to resolve pods. Therefore, in this new proposal, pod names will continue to be specified with a simple Str name instead of a Uri.
A rich standard Env library as an alternative
What if we were to build a library of Env classes to fill the same kinds of needs that URIs would fill?
The most important use case of pod URIs is to allow Fan scripts to "import" other scripts. The most common case is where you want to load scripts in the same directory. I threw together a prototype called ScriptEnv last week which does exactly this:
// foo.fan
using hello
class Foo
{
static Void main()
{
Hello.greet
}
}
Running fan foo.fan with ScriptEnv as the environment does exactly what you expect, echoing Hello world! to the screen.
There are some limitations to this. With my particular implementation of ScriptEnv, the scripts have to reside in the current directory. But it's easy enough to generalize. You could create a ScriptPathEnv class to search a directory list for the desired pod.
That's actually the beauty of delegating the "where" entirely to envs. Users are always free to create alternatives and the cost of doing so is very low. We can experiment and "grow" a standard set of environments which solve 99% of our user's needs instead of hardcoding it into the fan URI scheme.
Another case highlighted by Brian and Andy (in ticket #720) is loading pod files from scripts that were stored on a database instead of a filesystem. It's very easy to create a new DatabaseEnv to do this. It's identical to ScriptEnv, except that it searches a database instead of a directory and loads the file from a SQL blob instead of from a file.
The really cool feature with Envs is that they are so incredibly easy to swap out. Say we were using ScriptEnv to prototype the example above. Now that we have perfected our hello pod, we want to commit it to our database. To do this, we just dump the contents of hello.fan into a row of our database. We swap over to DatabaseEnv and with no other changes, our code runs.
A WebEnv environment which pulls pods over a trusted network would create a very cloud-computer-y feeling for developers. The servername, remote path, authentication details (the "where") all belong in the WebEnv object instead of being spread out throughout the entire codebase's using statements.
Env chaining and parametrizing
Eventually, the Fantom tools will make it very easy to hook into the full power of environments. Right now, all you can do is specify which Env class to instantiate. A generic instance is created from the default constructor. In the future, it will be possible to chain and parametrize environments.
Parametrizating would mean specifying the username, password, and host of your DatabaseEnv or WebEnv, or specifying the "include" directories of your ScriptEnv (or your repo path for PathEnv).
Env chaining is specifying more than one Env at a time. Environments will delegate unknown pods to their parents in the chain. The top of the chain is always sys::BootEnv.
For example, take the chain:
DatabaseEnv(ScriptEnv(BootEnv()))
We want to search for the hello pod. We call Env.cur.findPodFile("hello"). DatabaseEnv would first search the database. If hello was not in the database, ScriptEnv looks in the current directory. If it's not found there either, we check the installed pods.
It becomes so easy to relocate pods that the "where" of a pod becomes secondary.
The fan: URI scheme
Brian and Andy are really keen on using Uris for just about everything. I am very happy to have them so well integrated into the language, and I appreciate how easy it is to do webby things with them. However, I don't feel this is the right place to use them.
For pods, the pod::Type.slot naming scheme is just too perfect. It's simple, easy to parse, and it follows the Perl tenent of "different things should look different".
Pods can still in some sense be named with Uris, but these Uris should be kept as a detail to a particular environment. As I hope I've illustrated, pods can be loaded in all sorts of ways, and to me, it feels terribly limiting to force all Envs to conform to a single namespace convention.
Script imports
I realize that not being able to use relative paths in a using statement is an inconvenience. I could see this as the one special case where the rules should be broken and a file:-based Uri should be allowed. But only because it is the humane thing to do ;)
Foreign Function Interface
This is a truly interesting case. Specifying an FFI is half-way between "what" and "where". Take [java]javax.swing for example. It's a "what" because it IS a Java type. But it's also a "where" because it loads in a totally different manner than the other pods.
I'm not 100% sure where it might belong in my proposal. Should it be routed through the current environment like everything else? If an env would handle it, it would have to be pretty low-level stuff, knowing the internals of the JVM or .NET.
Still, there might be some interesting consequences of extending sys::Env to handle FFI lookups. For [java], you could create Envs which modify the classpath. You could also have envs which load classes over a secure network or that modify a configuration file or check the security manager or any number of other cool things.
Dotted pod names
One tacit assumption I've made here is that pod names are unique. This assumption is backed by the documentation in docLang. As far as I understand, #934 did not attempt to tackle possible issues with pod name uniqueness.
However, jodastephen raised an important argument in topic `http://fantom.org/sidewalk/topic/957#c7182`. A large business may grow a large number of pods over time. It would be in their best interest to make it easy for them to organize all their pods in a hierarchy internally.
I would like to propose as a simple addendum that we allow pod names to be dotted. For example, net.tac-tics.hello. Dots are already legal in the FFI pod names (such as [java]javax.swing). I'm not proposing a packaging system like Java has. Just the simply ability for pod names to contain dots according to the grammar: <id> ("." <id>)*. These dotted section may optionally be handled specially by the loading environments. This way, a business with a large codebase gains a ton of control over how pods are loaded, able to delegate entire "sub-pods" to other environments.
Finale
So there you have it. I took a long time to come up with this, so please give it due consideration.
Brian and Andy, I know you have some strong ideas on how you want to do this already. To that, my best appeal is that the above is the simplest thing that could possibly work. It requires zero changes to the compiler or APIs. It is merely a policy for dealing with these kinds of issues: that they are to be delegated to new subclasses of Env. Lastly, new developers are not confronted with the awkward URI conventions (things like fan:[java:javax.swing]::JFrame) and their corner cases.
As always, let me know what you guys think :)
mslMon 8 Mar 2010
Hear! Hear!
The inclusion of paths/Uris always felt a bit wrong to me and I think you've nailed that gut feeling here with your where/what differentiation!
A couple of thoughts (nowhere near as well thought through as yours):
I'd be inclined to have java FFI go through the same process as everything else to be looked up (ie a chained Env that somewhere along the line talks classloader). In this case the [java] qualifier may become redundant, or it may be a generic way to express a preference for which Env you want a class from (or to be tried first)
I wonder whether expanding your dot notation suggestion to include / along with my suggestion above for preferring Envs might get further than adding a special case for handling relative paths? You could theoretically allow something like using [file]../include/Included.fan and the file Env will solve the rest (looking at it here it feels quite clunky - I'm sure there's something more elegant)
brianMon 8 Mar 2010
very nice writeup
One important thing, which I thought I've already mentioned - regardless of what we do with Uris for pods, the plan is to keep the existing "pod::Type.slot" naming syntax.
So the only issue is really what is the mini-syntax of the pod itself. Today it is a valid Fantom identifier or FFI. The reason I don't really like that is that has required some name mangling for file scripts. And then we have a slightly different syntax for Java FFI (but the fact that is does allow dots is problematic).
The basic idea of your proposal that the using statement should define what and not where seems quite sound. In fact, this was my basic philosophy for pods with environments - that the Env should know where things are not the depends/using. So I think it makes perfect sense to try and apply that principle to scripts. So I think it is an excellent idea to pursue.
Here is the really tricky problem with scripts. To identify what means the best design pattern is to identify an imported script with just the filename minus the extension (sort of like Python). However, the actual pod name of a script can never be just the filename because it doesn't guarantee uniqueness with-in the VM, which is why today scripts get assigned a podName of the entire canonical path mangled into an identifier. I don't think that is an unsolvable problem, but it does require some mechanism for scoping the script environments and names in a consistent way.
lbertrandMon 8 Mar 2010
@tactics
Well done in putting into word what was always feeling wrong - distinction between what and where is the way forward and need to always be kept in mind...
KevinKelleyMon 8 Mar 2010
@tactics - nice, I think I agree with almost all of that.
A thing that bothered me about the Uri usings is that it feels like old C-style #include syntax, and I don't mean that in a good way. :-) "What it is, not where it is", good phrase.
But, there's a need for some mechanism to perform the function. I'm not entirely sure anything new is needed at this point -- I mean, it feels right to me, that Env encapsulates this "execution environment" concept of handling the loading of arbitrary code into a shared namespace.
So we need to play around with some different Envs.
ScriptEnv, that initializes itself in a directory, loads files as podname.fan (with hot-reload!). Here a file "is" a pod, and can have many types in it.
ScriptPathEnv, init with a root dir, loads subdirs as podname/srcfiles.fan.
I hesitate to bring up dotted names... too easy to lock in the wrong thing, like Java, where the dot might as well be an underscore for all the good it does. In fact I think a ScriptPathEnv that translated underscores in the podname to / on the filesystem would give the effect of Java's "packages".
Anyway, yeah, this sounds like how Env ought to work.
katoxMon 8 Mar 2010
+1 on this proposal. I could not digest #934 somehow but I could not point anything particulary bad either. I'm really glad for such a nice writeup. I'm not sold on dotted pods though.
Another interesting scenario comes back. With Env smart enough we could even change implementation pods without an hassle. using log::Logger could be easily mapped to whatever implementation (in the whole project env) if the interface is the same without confusing anyone or breaking anything. The only blocking thing I see is that the original pod name might be used reflection code with a different pod identifier.
lbertrandMon 8 Mar 2010
I just realised that can also address a question I was having regarding having the same pod in different version in the path... A specific Env can also resolve the right pod using the dependencies declared in a pod, so that each pod can really use the right version of the dependency.
tacticsTue 9 Mar 2010
@all Thanks for all the feedback. It sounds like there were a lot of unsure feelings towards #934.
@brian Naming pods generated from scripts is indeed tricky. Script files aren't in any sense "installed" and they are fairly anonymous.
If we use a custom Env to load scripts, this isn't a problem. Or rather, it's a problem for the Env to deal with.
If we allow local file Uris in using statements as per my Script imports section above, we need to more carefully consider how to name them. If two scripts want to import the same third script, the pod name must be the same even if the Uris are different. So util.fan needs to have the same pod name throughout in this example:
// /home/tactics/foo.fan
using `includes/util.fan`
class Foo { ... }
.
// /home/tactics/includes/bar.fan
using `util.fan`
class Bar { ... }
.
// /home/tactics/includes/util.fan
class Util { ... }
Perhaps, in the script case, we could default the podname to the script name, but allow for the pod to be renamed with the as keyword:
// /home/tactics/foo.fan
using `includes/util.fan` as utilPod
class Foo { ... }
If we allowed as here, we might even let the local filepath be a directory:
// /home/tactics/foo.fan
using `includes/` as utilPod // /includes/*.fan are compiled into utilPod
class Foo { ... }
Just a thought. I think it's best to keep the script usings as simple as possible and let Env take center stage.
Also, in another thread, you brought up the issue of where configuration files live. While much of that detail can be decided by individual Envs, I agree this needs to be discussed. It would be nice to have a general set of guidelines.
@lbertrand Writing a special Env class to do version checks would be an interesting possibility. You could then easily do backwards-compat testing by modifying the env-chain.
@katox Switching pod implementations is also a really interesting use. The tricky part is finding two pods that provide the same API. But I'm sure it would be a good hack for some people after they've painted themselves into a corner :)
@msl I think the [ffi] modifiers are good to keep as the difference between a Fan pod and a Java pod is a big one. It might be nice to split the ffi name out when passing it to the Env, though:
Or something like that. The API for Env needs to be extended if we are going to have it handle FFI.
@KevinKelley What's with all the dot hate? :P
I think you idea of hotswapping pods might be something very cool to look at in the future.
KevinKelleyTue 9 Mar 2010
What's with all the dot hate?
Mostly this:
Although packages lower in the naming hierarchy are often referred to as "subpackages" of the corresponding packages higher in the hierarchy, there is no semantic relationship between packages
JSR 277 and 294 and OSGI are attempts to address issues of packaging, or modules, in Java; the existing "system" is just, "give everything a long complicated name, and dump it all together (with classpath) and hope it works."
Java has something called "packages", but they're not packagy.
It's not really "hate"; we don't need anything fancy, just this is a case where "do what java did" is a bad idea. Thus, my "no dotted names". Until we make it mean something sensible.
tacticsTue 9 Mar 2010
I see you concern.
Rest assured that my intention was only to allow dots in the identifiers. The semantics are left entirely unspecified and that's on purpose. So I could just as easily call my package tacticsFoo or tactics.foo or tacticsDOTfoo. To the compiler and runtime, it's just a name.
However, the current environment is allowed to designate a meaning to those dots. It could be something like Java. It could be something much more intelligent. But it's always up to the environment.
Furthermore, my idea was that support for dotted names would not be part of the core language. The standard library would remain as it is. The tutorials will all say to use regular, camel-case pod names: fooBarBaz. For ${Float.random}% of the users, the dots will be unused or possibly unknown.
However, if an organization has the desire to organize their codebase in a hierarchial fashion, the dots will be there. It's up to them to write their own Env to support it. Instead of writing divisionDepartmentGroupProjectPodname you can spell it as division.department.group.project.podname, which is much easier to parse both with the eyes and with the code.
(And lastly, if they do end up using dotted names, we can teach new Fanners to do away with the atrocious com.example.whatever convention ;)
EDIT: I forgot to mention. As a bonus, allowing dots in podnames is a small unification. Since both .NET and Java allow dots in their packaging system, we don't have to allow dots as a weird exception to the rules.
katoxTue 9 Mar 2010
However, the current environment is allowed to designate a meaning to those dots.
Wouldn't this cause possible compatility issues? If you created a pod in one Env but you wanted to use it later in a different Env you might end up having to rename everything to follow some conventions imposed by the deployment environment.
tacticsTue 9 Mar 2010
you might end up having to rename everything
If moving from an unstructured pod system (no dots) to a structured one (with dots), there might be some migration required. But it's the cost of admission for organizing your code.
Once your codebase is converted to using dots, though, it should be easy to tweak the underlying Env.
mslTue 9 Mar 2010
WRT dots in pod names - you can achieve the same effect right now with underscores rather than dots. It has the same outcome (nothing to the system, only makes names more readable and somewhat hierarchical). Admittedly, it is a bit of an eyesore (I'd much rather look at com.example.whatever than com_example_whatever) but I suspect that's just a learned preference.
Personally, I wouldn't be particularly fussed not having access to dots, other than for consistency with java/.NET/python if all it's intended for is user friendliness.
As to the semantics of these dots and issues with switching between Env instances, again there's no reason you couldn't do that right now. Let's say I need using xml - there's no reason you couldn't have an Env which instead of loading the standard fantom xml pod went off and picked up some super duper xml pod (that say supported schema validation) from somewhere else and returned that. Then anything that you used that ever needed XML would magically get validation without changing.
It may well be considered bad form to swap this out without the user being aware (suddenly a bunch of pods you use that didn't care about valid schemas start breaking), but I can also see cases where it could be very useful (for example providing a dev vs prod build of the same pod, or augmenting the fcode returned in a pod on the fly to achieve some form of aspect oriented programming).
KevinKelleyTue 9 Mar 2010
But as soon as you allow dots into Fantom code ( using dotted.pod.name ) then all Fantom code has to deal with what it means... unless it means nothing special at all; in which case might as well use underscores or camel.
My earlier point is that dotted pods could end up having useful meaning: nested modules, or subsystems, or something, so don't lock the language into the "dots are just pretty" meaning of Java packages.
Letting Env manage its internals is good, but a DerivedEnv would have to export its symbols in a way that's understood by the rest of the system. tactics, your idea is good, but I don't think it addresses enough of the module issues yet, so I don't want to lock down the naming convention to a least-common-denominator meaning.
Here's a slide presentation on modules in Groovy and OSGI -- hits some highlights on modules as opposed to naked jars.
I don't find anything to follow up on Brian's script namespace comments, but that's an issue too.
tacticsWed 10 Mar 2010
But dots ARE pretty :)
You raise a good issue about reserving them for the future. I can always bring up my "dots are pretty" idea down the road.
brianWed 10 Mar 2010
A pod name is just an opaque name in a global flat namespace. Java packages by the way are exactly the same. The reasons dots are so problematic is that dots have a very explicit meaning in the language as the Type.slot separator.
This is why pod names are identifiers today - because other than readability, there is no pod hierarchy, and never will be one - pods will always be an opaque name at the top of the namespace.
That doesn't negate the desire for a consistent way to define compound names. Today camel case is convention. Maybe that isn't readable for super long names, but I think using super long names is to be discouraged, just like AbstractManagerFactoryFactory is a poor type name and hard to read.
A better way to think of dots or any general purpose separator char is to think about it more like an escape sequence. For example suppose I want to import the script pod "foo-bar.baz.fan"? Is that even allowed? If so, how do I design a syntax that works in both using statements and fully qualified type names?
Although this problem was solved by Java FFI in using statements, you can't use a fully qualified Java FFI type in normal code - one of the problems which must be solved by the eventual design. My URI proposal was designed to handle this problem by using an actual URI literal to simplify the grammar and provide a completely open set of chars for pod naming. That problem doesn't go away:
what is valid chars in pod name?
how are these chars escaped/formatted in using statement and fully qualified names
ideal syntax enables pod literals
But I don't want to get too focused on the dot vs naming convention issue. To move the ball forward we need to focus on big picture stuff:
think more deeply about Env chaining and what it means
think about how Envs scope their podname (filename versus canonical path)
think about how reloading of scripts changes names and dependencies
define standardized rules for how scripts should be load/managed
tacticsWed 10 Mar 2010
While writing this thread, I noticed that Env chaining is actually kind of weird. I ended up cheating a little bit when describing it so I wouldn't bog down the discussion with technicalities.
The issue is that Envs have no responsibility to delegate in any predictable way. Generally, there are three common cases:
An Env handles all cases ( BootEnv )
An Env tries to handle each case, but delegates to its parent on failure ( PathEnv )
An Env delegates each case to its parent, and handles it itself if that fails. (The actual version of my ScriptEnv)
I wrote ScriptEnv to delegate to its parent first so that script files could not shadow installed pods. I didn't realize until afterwards that this makes chaining less intuitive. If my chain looks like this:
DatabaseEnv(ScriptEnv(BootEnv()))
Pods are actually looked up in this order:
DatabaseEnv
BootEnv
ScriptEnv
There are a few ways we can handle this:
Allow this weird ordering, giving each Env in the chain total control of how things are loaded.
Require that Envs always delegate after a lookup fails as a convention.
Require that Envs always delegate after a lookup fails by making the parent slot private and having the runtime manage delegation.
I also mentioned in the Env chaining and parametrizing section that BootEnv should always be at the top of the chain. Rethinking this, it probably isn't a good idea. If I rewrite ScriptEnv to delegate after a lookup fails, all I need to do is set my chain to be:
Database(BootEnv(ScriptEnv()))
And the lookup order is the same:
DatabaseEnv
BootEnv
ScriptEnv
katoxWed 10 Mar 2010
I wrote ScriptEnv to delegate to its parent first so that script files could not shadow installed pods. I didn't realize until afterwards that this makes chaining less intuitive.
The ability to prevent a custom classloader to load system classes may not be very intiutive but it is important for security reasons. See for instance, GF Classloader hierarchy.
However note that the Servlet Specification (10.7.2 of version 3.0) recommends the opposite "It is recommended also that the application class loader be implemented so that classes and resources packaged within the WAR are loaded in preference to classes and resources residing in container-wide library JARs.".
Still security is preserved by restricting a conforming web application classloader to "Java EE product should not allow the application to override Java SE platform classes, such as those in the java. and javax. namespaces, that Java SE does not allow to be modified. The container should not allow applications to override or access the container’s implementation classes."
Web application classloader behaviour is usually configurable and must be set to a certain value to support underlying technologies (like EJB modules).
What this all means is probably that there is no easy one fits all solution and that a great amount of flexibility is really needed.
tompalmerThu 11 Mar 2010
I've been AWOL, sorry. A mix of too busy and doing my own things. In my more recent thoughts over the past year, I'm moving towards liking an AOP way of dealing with imports.
That is, I don't really want to specify import/using statements in my individual source files at all. I'd rather the dependency setup for a project (pod config, path environment variable, classpath, script tags in HTML, startup scripts in MATLAB, ...) configure which packages/pods/namespaces/modules will be active in my code. And I just write code assuming what I need is available.
As in, "Look, Ma, no imports!"
Only in the rare case of conflicting names being needed in the same project would I need to resolve different namespaces at the individual file (or subfile) level. When that happens, maybe clarifying it right in the file would be best. I'm not 100% sure, though.
On performance issues: Resolving names when not specified per file could be a bit slower for compiling, but I think a simple hash index in memory would cover most cases, and fancier solutions could involve disk-based indexing.
tacticsThu 11 Mar 2010
That is, I don't really want to specify import/using statements in my individual source files at all.
This is an interesting notion.
Still, I think usings are a powerful form of self-documentation. If you see an Event class used in a source file, you would have to do some research to discover whether it was fwt::Event, dom::Event, or an Event class in the current pod.
brianThu 11 Mar 2010
Actually I really like the idea of treating a directory (or some group of files) as a single pod. That might simplify how hot-reload is done when you have dependencies between script files.
tcolarThu 11 Mar 2010
While I like tactics proposal, I'm wondering if that wouldn't make tooling very difficult since it will have to figure out/use whatever Env would be used at runtime.
As far the idea of treating a directory (or some group of files) as a single pod, that obviously would keep really easy for tools/IDE to deal with.
tacticsThu 11 Mar 2010
For lookups, tools could just delegate to a live env chain.
I agree grouping multiple scripts into a single pod would be a good idea if we want to do hot loading.
katoxThu 11 Mar 2010
I like the idea of no import statements - all dependencies handled via pods and Env, even in scripts.
I can't think of a case in Java when I really wanted to write some imports. I declare all dependencies for each subsystem in maven anyway why should I bother again? The only outcome is probably that I have to resolve import conflicts (among commiters) when we change some generic implementation to our own flavour.
Any decent tooling should be able to display pod origin of any class. I don't see a way how could I judge directly if Event class comes from dom or fwt - could be either of them - there is no import of each class. It is the same information as I can get from declared pod dependencies.
using as is comfortable but as in Java I could live as well with fully qualified names for such rare cases.
jodastephenFri 12 Mar 2010
No imports sounds appealing. I think you'd have to deal with the ambiguities though. Maybe it as imports setup on a per pod basis.
tcolarFri 12 Mar 2010
I don't think I like No import for several reasons:
When working on a large code base, as I do at work, it's very common that i go look at the import statement to find where a type came from, so I can go llok at it.
It's very common to have types with names like Event, Action or BusinessObject in many places, even the Fantom distro as many duplicated type names (plus thefact that all the ones in sys are implied), so that would just make it that much harder to find which is being used.
Mots(if not all) modern languages like ruby, groovy or scala all use them, there must be a reason.
tacticsFri 12 Mar 2010
Mots(if not all) modern languages like ruby, groovy or scala all use them, there must be a reason.
Scripting languages don't have a build process and don't have any centralized way of naming dependencies. If they don't declare it per-file, they can't declare it at all.
However, Fantom lives in a weird limbo space. It wants to be both a compiled language AND a scripting language. (I really like that aspect of the language).
Any decent tooling should be able to display pod origin of any class.
Are you saying Notepad++ and vim aren't decent tools? :) (Not to mention Andy and Brian have said they aren't heavy IDE users).
Plus, there are lots of times when you're browsing code in a way that an IDE can't help you. For example, browsing BitBucket.
brianFri 12 Mar 2010
I lean towards keeping using statements as they are for inter-pod imports. It isn't all that onerous even though its a bit anti-DRY with the pod depends. But has the benefit of being a little more explicit in readability of how source file's namespace is being managed (in case you copy files or change your pod dependencies).
However its a different issue if multiple script files should be bundled into one logical pod, or treated as individual pods. I'm really digging the idea of bundling them into one pod (in which case there is no using statement b/c they all live in the same pod).
I think we if also say that a "script pod" can't depend on other script pods, then we have a clean mechanism for auto-reload whenever one file changes in a script bundle. Not sure how that might effect common situations where you want a chrome.fan script to be used by multiple page scripts:
chrome.fan
page1.fan
page2.fan
Ideally you'd want pod page1 to be composed of "chrome.fan" and "page1.fan". The page2 pod would be composed of "chrome.fan" and "page2.fan". Then:
"chrome.fan" changed -> both page1 and page2 pods would be auto reloaded
"page1.fan" changes -> page1 pod is reloaded
"page2.fan" changes -> page2 pod is reloaded
Not sure how different scripting languages like Ruby work, but they are monkey patched at runtime so its a lot simpler to handle than JVM classes.
katoxFri 12 Mar 2010
@tactics
Are you saying Notepad++ and vim aren't decent tools? :)
Well... I supplied vim syntax rules and still use it as my primary working setup so clearly this is the time when Matlock would object ;).
I can ask again how using fwt and using dom would help if you used an Event. In Java - OK - there will probably be an import of this class. But in Fantom there are only two lines (at best) - it could come from any those or it could be implicit.
If there was nothing you'd have to go to pod declaration but so what? If you don't know any context of the source file it is not your biggest problem anyway (what about classes defined in other pod files etc). If you do know the context you probably know it comes from fwt or dom pod (previous experience).
I understand your fear - I am really not comfortable with a bunch of import static statements in Java - but in Fantom using statement is actually a quite different beast.
@tcolar
When working on a large code base, as I do at work, it's very common that i go look at the import statement to find where a type came from, so I can go look at it.
You can look it up in your pod dependencies. The information is the very same (unlike in Java where you can find individual classes).
names like Event, Action ... make it that much harder to find which is being used
using fwt is helpful in what regard exactly?
there must be a reason.
I don't see it right now. I would appreciante the debate to reveal it, honestly. BTW Newspeak doesn't use them - G. Bracha left it to tooling.
KevinKelleySat 13 Mar 2010
There's something to be said for limiting the universe of discourse... A larger project will depend on a quite a few pods. But each source file should not. Growing lists of usings is a good indication that you're losing cohesion in your design.
Being able to narrow your focus to a particular "thing", whatever that thing may be, helps you to understand it and make it right.
using fwt is helpful in what regard exactly?
using fwt tells me that this is user-interface. Narrowing it down further by using the individual classes you need, is even better.
Better tooling mitigates; with completion and context-popups you end up wanting usings to be auto-managed. But I don't think they should go away... the best tool will probably always be a brain and an editor.
andySat 13 Mar 2010
However its a different issue if multiple script files should be bundled into one logical pod, or treated as individual pods
So how do you define which scripts get included? We would need something flexible enough to define pulling in scripts from a database or arbitrary directories - but also something that works out-of-the-box using all scripts in the current directory maybe.
Maybe that is just the default Env - and you can plug in a more complex Env for the database issue.
Either way this (single script pod) seems like a good middle ground between scripting and pods. We make scripting flexible enough to span multiple files (which we need) - but don't go overboard when we have a formal way to tackle that in pods.
tactics Mon 8 Mar 2010
Summary
Environments are a very powerful extension of the Fantom language. I think they have many potential uses and I'd like to see them well supported by the rest of the language.
I want to propose an alternative to ticket Uri Namespace (#934). I believe many of the problems that proposal deals with can be resolved by clever use of environments. Arguably, of course, in a cleaner, more modular way.
The gist of this alternative proposal is:
pod::Type.slot
syntax for naming Fantom entities, andsys::Env
solely responsible for resolving simple pod names to actual pods.Background
The problems this proposal and ticket #934 intend to solve are both fairly involved and leave a lot of room for debate. Rather than bog down the #934 thread, I wanted to fork it to propose a possible alternative here.
To summarize, #934 proposes three separate changes:
sys::UriSpace
classusing
statements to use URIsI don't think anyone really misses
sys::UriSpace
all that much, so it will not be mentioned again.As I've previous stated in the #934 thread, I don't like the URI syntax proposed for describing the pods. The
pod::type.slot
syntax is so impossibly simple. I feel it is the Right Thing to do.My main argument is that the current environment should be the one and only part of the system which cares about where pod files are loaded from.
Using as "what" not "where"
What is the meaning of the
using
statement?Currently, the
using
statement tells you "what" pod dependency exists for a Fantom source file. If you want to use any class from a pod, you must declare it as a dependency up at the top of your file.Ticket #934 plans to change that meaning. According to the proposed changes, the
using
statement tells you "where" a dependency exists. You must specify the URI pointing to the pod you want.According to my new proposal, the
using
statement remains a "what". The "where" of a dependency belongs exclusively to the currentEnv
class. TheEnv
class is also promoted in importance across the language.Pod URIs and Envs are at odds
The problem with naming pods with URIs is that URIs often contain information about "where" a resource exists. With that comes assumptions about how a resource is to be resolved. For example, take a class named
fan:[file:../shared/script.fan]::SomeClass
(according to #934's URI conventions). There is a strong assumption thatSomeClass
's pod will be loaded from a file namedscript.fan
. But ultimately, the decision of "where" a pod is loaded from rests with the current environment.By allowing the location of a pod to be specified in two places, you create some bad redundancy. Most likely, the URI will end up "winning" and the usefulness of the
Env
mechanism is substantially diminished.Your class doesn't really care "where" a pod comes from anyway. It just wants it to be there when the time comes. The environment is the only one who cares about "where", and it should be granted full authority on how to resolve pods. Therefore, in this new proposal, pod names will continue to be specified with a simple
Str
name instead of aUri
.A rich standard Env library as an alternative
What if we were to build a library of
Env
classes to fill the same kinds of needs that URIs would fill?The most important use case of pod URIs is to allow Fan scripts to "import" other scripts. The most common case is where you want to load scripts in the same directory. I threw together a prototype called
ScriptEnv
last week which does exactly this:.
Running
fan foo.fan
withScriptEnv
as the environment does exactly what you expect, echoingHello world!
to the screen.There are some limitations to this. With my particular implementation of
ScriptEnv
, the scripts have to reside in the current directory. But it's easy enough to generalize. You could create aScriptPathEnv
class to search a directory list for the desired pod.That's actually the beauty of delegating the "where" entirely to envs. Users are always free to create alternatives and the cost of doing so is very low. We can experiment and "grow" a standard set of environments which solve 99% of our user's needs instead of hardcoding it into the
fan
URI scheme.Another case highlighted by Brian and Andy (in ticket #720) is loading pod files from scripts that were stored on a database instead of a filesystem. It's very easy to create a new
DatabaseEnv
to do this. It's identical toScriptEnv
, except that it searches a database instead of a directory and loads the file from a SQL blob instead of from a file.The really cool feature with Envs is that they are so incredibly easy to swap out. Say we were using
ScriptEnv
to prototype the example above. Now that we have perfected ourhello
pod, we want to commit it to our database. To do this, we just dump the contents ofhello.fan
into a row of our database. We swap over toDatabaseEnv
and with no other changes, our code runs.A
WebEnv
environment which pulls pods over a trusted network would create a very cloud-computer-y feeling for developers. The servername, remote path, authentication details (the "where") all belong in theWebEnv
object instead of being spread out throughout the entire codebase'susing
statements.Env chaining and parametrizing
Eventually, the Fantom tools will make it very easy to hook into the full power of environments. Right now, all you can do is specify which
Env
class to instantiate. A generic instance is created from the default constructor. In the future, it will be possible to chain and parametrize environments.Parametrizating would mean specifying the username, password, and host of your
DatabaseEnv
orWebEnv
, or specifying the "include" directories of yourScriptEnv
(or your repo path forPathEnv
).Env chaining is specifying more than one Env at a time. Environments will delegate unknown pods to their parents in the chain. The top of the chain is always
sys::BootEnv
.For example, take the chain:
We want to search for the
hello
pod. We callEnv.cur.findPodFile("hello")
.DatabaseEnv
would first search the database. Ifhello
was not in the database,ScriptEnv
looks in the current directory. If it's not found there either, we check the installed pods.It becomes so easy to relocate pods that the "where" of a pod becomes secondary.
The fan: URI scheme
Brian and Andy are really keen on using
Uri
s for just about everything. I am very happy to have them so well integrated into the language, and I appreciate how easy it is to do webby things with them. However, I don't feel this is the right place to use them.For pods, the
pod::Type.slot
naming scheme is just too perfect. It's simple, easy to parse, and it follows the Perl tenent of "different things should look different".Pods can still in some sense be named with
Uri
s, but theseUri
s should be kept as a detail to a particular environment. As I hope I've illustrated, pods can be loaded in all sorts of ways, and to me, it feels terribly limiting to force allEnv
s to conform to a single namespace convention.Script imports
I realize that not being able to use relative paths in a
using
statement is an inconvenience. I could see this as the one special case where the rules should be broken and afile:
-basedUri
should be allowed. But only because it is the humane thing to do ;)Foreign Function Interface
This is a truly interesting case. Specifying an FFI is half-way between "what" and "where". Take
[java]javax.swing
for example. It's a "what" because it IS a Java type. But it's also a "where" because it loads in a totally different manner than the other pods.I'm not 100% sure where it might belong in my proposal. Should it be routed through the current environment like everything else? If an env would handle it, it would have to be pretty low-level stuff, knowing the internals of the JVM or .NET.
Still, there might be some interesting consequences of extending
sys::Env
to handle FFI lookups. For[java]
, you could createEnv
s which modify the classpath. You could also have envs which load classes over a secure network or that modify a configuration file or check the security manager or any number of other cool things.Dotted pod names
One tacit assumption I've made here is that pod names are unique. This assumption is backed by the documentation in docLang. As far as I understand, #934 did not attempt to tackle possible issues with pod name uniqueness.
However, jodastephen raised an important argument in topic `http://fantom.org/sidewalk/topic/957#c7182`. A large business may grow a large number of pods over time. It would be in their best interest to make it easy for them to organize all their pods in a hierarchy internally.
I would like to propose as a simple addendum that we allow pod names to be dotted. For example,
net.tac-tics.hello
. Dots are already legal in the FFI pod names (such as[java]javax.swing
). I'm not proposing a packaging system like Java has. Just the simply ability for pod names to contain dots according to the grammar:<id> ("." <id>)*
. These dotted section may optionally be handled specially by the loading environments. This way, a business with a large codebase gains a ton of control over how pods are loaded, able to delegate entire "sub-pods" to other environments.Finale
So there you have it. I took a long time to come up with this, so please give it due consideration.
Brian and Andy, I know you have some strong ideas on how you want to do this already. To that, my best appeal is that the above is the simplest thing that could possibly work. It requires zero changes to the compiler or APIs. It is merely a policy for dealing with these kinds of issues: that they are to be delegated to new subclasses of
Env
. Lastly, new developers are not confronted with the awkward URI conventions (things likefan:[java:javax.swing]::JFrame
) and their corner cases.As always, let me know what you guys think :)
msl Mon 8 Mar 2010
Hear! Hear!
The inclusion of paths/Uris always felt a bit wrong to me and I think you've nailed that gut feeling here with your where/what differentiation!
A couple of thoughts (nowhere near as well thought through as yours):
Env
that somewhere along the line talks classloader). In this case the[java]
qualifier may become redundant, or it may be a generic way to express a preference for whichEnv
you want a class from (or to be tried first)/
along with my suggestion above for preferringEnv
s might get further than adding a special case for handling relative paths? You could theoretically allow something likeusing [file]../include/Included.fan
and the fileEnv
will solve the rest (looking at it here it feels quite clunky - I'm sure there's something more elegant)brian Mon 8 Mar 2010
very nice writeup
One important thing, which I thought I've already mentioned - regardless of what we do with Uris for pods, the plan is to keep the existing "pod::Type.slot" naming syntax.
So the only issue is really what is the mini-syntax of the pod itself. Today it is a valid Fantom identifier or FFI. The reason I don't really like that is that has required some name mangling for file scripts. And then we have a slightly different syntax for Java FFI (but the fact that is does allow dots is problematic).
The basic idea of your proposal that the
using
statement should define what and not where seems quite sound. In fact, this was my basic philosophy for pods with environments - that the Env should know where things are not the depends/using. So I think it makes perfect sense to try and apply that principle to scripts. So I think it is an excellent idea to pursue.Here is the really tricky problem with scripts. To identify what means the best design pattern is to identify an imported script with just the filename minus the extension (sort of like Python). However, the actual pod name of a script can never be just the filename because it doesn't guarantee uniqueness with-in the VM, which is why today scripts get assigned a podName of the entire canonical path mangled into an identifier. I don't think that is an unsolvable problem, but it does require some mechanism for scoping the script environments and names in a consistent way.
lbertrand Mon 8 Mar 2010
@tactics
Well done in putting into word what was always feeling wrong - distinction between what and where is the way forward and need to always be kept in mind...
KevinKelley Mon 8 Mar 2010
@tactics
- nice, I think I agree with almost all of that.A thing that bothered me about the Uri usings is that it feels like old C-style #include syntax, and I don't mean that in a good way. :-) "What it is, not where it is", good phrase.
But, there's a need for some mechanism to perform the function. I'm not entirely sure anything new is needed at this point -- I mean, it feels right to me, that
Env
encapsulates this "execution environment" concept of handling the loading of arbitrary code into a shared namespace.So we need to play around with some different Envs.
ScriptEnv
, that initializes itself in a directory, loads files aspodname.fan
(with hot-reload!). Here a file "is" a pod, and can have many types in it.ScriptPathEnv
, init with a root dir, loads subdirs aspodname/srcfiles.fan
.I hesitate to bring up dotted names... too easy to lock in the wrong thing, like Java, where the dot might as well be an underscore for all the good it does. In fact I think a
ScriptPathEnv
that translated underscores in the podname to/
on the filesystem would give the effect of Java's "packages".Anyway, yeah, this sounds like how
Env
ought to work.katox Mon 8 Mar 2010
+1 on this proposal. I could not digest #934 somehow but I could not point anything particulary bad either. I'm really glad for such a nice writeup. I'm not sold on dotted pods though.
Another interesting scenario comes back. With
Env
smart enough we could even change implementation pods without an hassle.using log::Logger
could be easily mapped to whatever implementation (in the whole project env) if the interface is the same without confusing anyone or breaking anything. The only blocking thing I see is that the original pod name might be used reflection code with a different pod identifier.lbertrand Mon 8 Mar 2010
I just realised that can also address a question I was having regarding having the same pod in different version in the path... A specific Env can also resolve the right pod using the dependencies declared in a pod, so that each pod can really use the right version of the dependency.
tactics Tue 9 Mar 2010
@all Thanks for all the feedback. It sounds like there were a lot of unsure feelings towards #934.
@brian Naming pods generated from scripts is indeed tricky. Script files aren't in any sense "installed" and they are fairly anonymous.
If we use a custom
Env
to load scripts, this isn't a problem. Or rather, it's a problem for theEnv
to deal with.If we allow local file
Uri
s inusing
statements as per my Script imports section above, we need to more carefully consider how to name them. If two scripts want to import the same third script, the pod name must be the same even if theUri
s are different. Soutil.fan
needs to have the same pod name throughout in this example:.
.
Perhaps, in the script case, we could default the podname to the script name, but allow for the pod to be renamed with the
as
keyword:If we allowed
as
here, we might even let the local filepath be a directory:Just a thought. I think it's best to keep the script
using
s as simple as possible and letEnv
take center stage.Also, in another thread, you brought up the issue of where configuration files live. While much of that detail can be decided by individual
Env
s, I agree this needs to be discussed. It would be nice to have a general set of guidelines.@lbertrand Writing a special
Env
class to do version checks would be an interesting possibility. You could then easily do backwards-compat testing by modifying the env-chain.@katox Switching pod implementations is also a really interesting use. The tricky part is finding two pods that provide the same API. But I'm sure it would be a good hack for some people after they've painted themselves into a corner :)
@msl I think the
[ffi]
modifiers are good to keep as the difference between a Fan pod and a Java pod is a big one. It might be nice to split the ffi name out when passing it to theEnv
, though:Or something like that. The API for
Env
needs to be extended if we are going to have it handle FFI.@KevinKelley What's with all the dot hate? :P
I think you idea of hotswapping pods might be something very cool to look at in the future.
KevinKelley Tue 9 Mar 2010
Mostly this:
JSR 277 and 294 and OSGI are attempts to address issues of packaging, or modules, in Java; the existing "system" is just, "give everything a long complicated name, and dump it all together (with classpath) and hope it works."
Java has something called "packages", but they're not packagy.
It's not really "hate"; we don't need anything fancy, just this is a case where "do what java did" is a bad idea. Thus, my "no dotted names". Until we make it mean something sensible.
tactics Tue 9 Mar 2010
I see you concern.
Rest assured that my intention was only to allow dots in the identifiers. The semantics are left entirely unspecified and that's on purpose. So I could just as easily call my package
tacticsFoo
ortactics.foo
ortacticsDOTfoo
. To the compiler and runtime, it's just a name.However, the current environment is allowed to designate a meaning to those dots. It could be something like Java. It could be something much more intelligent. But it's always up to the environment.
Furthermore, my idea was that support for dotted names would not be part of the core language. The standard library would remain as it is. The tutorials will all say to use regular, camel-case pod names:
fooBarBaz
. For ${Float.random}% of the users, the dots will be unused or possibly unknown.However, if an organization has the desire to organize their codebase in a hierarchial fashion, the dots will be there. It's up to them to write their own
Env
to support it. Instead of writingdivisionDepartmentGroupProjectPodname
you can spell it asdivision.department.group.project.podname
, which is much easier to parse both with the eyes and with the code.(And lastly, if they do end up using dotted names, we can teach new Fanners to do away with the atrocious
com.example.whatever
convention ;)EDIT: I forgot to mention. As a bonus, allowing dots in podnames is a small unification. Since both .NET and Java allow dots in their packaging system, we don't have to allow dots as a weird exception to the rules.
katox Tue 9 Mar 2010
Wouldn't this cause possible compatility issues? If you created a pod in one
Env
but you wanted to use it later in a differentEnv
you might end up having to rename everything to follow some conventions imposed by the deployment environment.tactics Tue 9 Mar 2010
If moving from an unstructured pod system (no dots) to a structured one (with dots), there might be some migration required. But it's the cost of admission for organizing your code.
Once your codebase is converted to using dots, though, it should be easy to tweak the underlying
Env
.msl Tue 9 Mar 2010
WRT dots in pod names - you can achieve the same effect right now with underscores rather than dots. It has the same outcome (nothing to the system, only makes names more readable and somewhat hierarchical). Admittedly, it is a bit of an eyesore (I'd much rather look at
com.example.whatever
thancom_example_whatever
) but I suspect that's just a learned preference.Personally, I wouldn't be particularly fussed not having access to dots, other than for consistency with java/.NET/python if all it's intended for is user friendliness.
As to the semantics of these dots and issues with switching between
Env
instances, again there's no reason you couldn't do that right now. Let's say I needusing xml
- there's no reason you couldn't have anEnv
which instead of loading the standard fantom xml pod went off and picked up some super duper xml pod (that say supported schema validation) from somewhere else and returned that. Then anything that you used that ever needed XML would magically get validation without changing.It may well be considered bad form to swap this out without the user being aware (suddenly a bunch of pods you use that didn't care about valid schemas start breaking), but I can also see cases where it could be very useful (for example providing a dev vs prod build of the same pod, or augmenting the fcode returned in a pod on the fly to achieve some form of aspect oriented programming).
KevinKelley Tue 9 Mar 2010
But as soon as you allow dots into Fantom code (
using dotted.pod.name
) then all Fantom code has to deal with what it means... unless it means nothing special at all; in which case might as well use underscores or camel.My earlier point is that dotted pods could end up having useful meaning: nested modules, or subsystems, or something, so don't lock the language into the "dots are just pretty" meaning of Java packages.
Letting Env manage its internals is good, but a DerivedEnv would have to export its symbols in a way that's understood by the rest of the system.
tactics
, your idea is good, but I don't think it addresses enough of the module issues yet, so I don't want to lock down the naming convention to a least-common-denominator meaning.Here's a slide presentation on modules in Groovy and OSGI -- hits some highlights on modules as opposed to naked jars.
I don't find anything to follow up on Brian's
script namespace
comments, but that's an issue too.tactics Wed 10 Mar 2010
But dots ARE pretty :)
You raise a good issue about reserving them for the future. I can always bring up my "dots are pretty" idea down the road.
brian Wed 10 Mar 2010
A pod name is just an opaque name in a global flat namespace. Java packages by the way are exactly the same. The reasons dots are so problematic is that dots have a very explicit meaning in the language as the
Type.slot
separator.This is why pod names are identifiers today - because other than readability, there is no pod hierarchy, and never will be one - pods will always be an opaque name at the top of the namespace.
That doesn't negate the desire for a consistent way to define compound names. Today camel case is convention. Maybe that isn't readable for super long names, but I think using super long names is to be discouraged, just like
AbstractManagerFactoryFactory
is a poor type name and hard to read.A better way to think of dots or any general purpose separator char is to think about it more like an escape sequence. For example suppose I want to import the script pod "foo-bar.baz.fan"? Is that even allowed? If so, how do I design a syntax that works in both using statements and fully qualified type names?
Although this problem was solved by Java FFI in using statements, you can't use a fully qualified Java FFI type in normal code - one of the problems which must be solved by the eventual design. My URI proposal was designed to handle this problem by using an actual URI literal to simplify the grammar and provide a completely open set of chars for pod naming. That problem doesn't go away:
But I don't want to get too focused on the dot vs naming convention issue. To move the ball forward we need to focus on big picture stuff:
tactics Wed 10 Mar 2010
While writing this thread, I noticed that
Env
chaining is actually kind of weird. I ended up cheating a little bit when describing it so I wouldn't bog down the discussion with technicalities.The issue is that
Env
s have no responsibility to delegate in any predictable way. Generally, there are three common cases:Env
handles all cases (BootEnv
)Env
tries to handle each case, but delegates to its parent on failure (PathEnv
)Env
delegates each case to its parent, and handles it itself if that fails. (The actual version of myScriptEnv
)I wrote
ScriptEnv
to delegate to its parent first so that script files could not shadow installed pods. I didn't realize until afterwards that this makes chaining less intuitive. If my chain looks like this:Pods are actually looked up in this order:
There are a few ways we can handle this:
Env
in the chain total control of how things are loaded.Env
s always delegate after a lookup fails as a convention.Env
s always delegate after a lookup fails by making theparent
slot private and having the runtime manage delegation.I also mentioned in the Env chaining and parametrizing section that
BootEnv
should always be at the top of the chain. Rethinking this, it probably isn't a good idea. If I rewriteScriptEnv
to delegate after a lookup fails, all I need to do is set my chain to be:And the lookup order is the same:
katox Wed 10 Mar 2010
The ability to prevent a custom classloader to load system classes may not be very intiutive but it is important for security reasons. See for instance, GF Classloader hierarchy.
However note that the Servlet Specification (10.7.2 of version 3.0) recommends the opposite "It is recommended also that the application class loader be implemented so that classes and resources packaged within the WAR are loaded in preference to classes and resources residing in container-wide library JARs.".
Still security is preserved by restricting a conforming web application classloader to "Java EE product should not allow the application to override Java SE platform classes, such as those in the java. and javax. namespaces, that Java SE does not allow to be modified. The container should not allow applications to override or access the container’s implementation classes."
Web application classloader behaviour is usually configurable and must be set to a certain value to support underlying technologies (like EJB modules).
What this all means is probably that there is no easy one fits all solution and that a great amount of flexibility is really needed.
tompalmer Thu 11 Mar 2010
I've been AWOL, sorry. A mix of too busy and doing my own things. In my more recent thoughts over the past year, I'm moving towards liking an AOP way of dealing with imports.
That is, I don't really want to specify import/using statements in my individual source files at all. I'd rather the dependency setup for a project (pod config, path environment variable, classpath, script tags in HTML, startup scripts in MATLAB, ...) configure which packages/pods/namespaces/modules will be active in my code. And I just write code assuming what I need is available.
As in, "Look, Ma, no imports!"
Only in the rare case of conflicting names being needed in the same project would I need to resolve different namespaces at the individual file (or subfile) level. When that happens, maybe clarifying it right in the file would be best. I'm not 100% sure, though.
On performance issues: Resolving names when not specified per file could be a bit slower for compiling, but I think a simple hash index in memory would cover most cases, and fancier solutions could involve disk-based indexing.
tactics Thu 11 Mar 2010
This is an interesting notion.
Still, I think
using
s are a powerful form of self-documentation. If you see anEvent
class used in a source file, you would have to do some research to discover whether it wasfwt::Event
,dom::Event
, or anEvent
class in the current pod.brian Thu 11 Mar 2010
Actually I really like the idea of treating a directory (or some group of files) as a single pod. That might simplify how hot-reload is done when you have dependencies between script files.
tcolar Thu 11 Mar 2010
While I like tactics proposal, I'm wondering if that wouldn't make tooling very difficult since it will have to figure out/use whatever Env would be used at runtime.
As far the idea of treating a directory (or some group of files) as a single pod, that obviously would keep really easy for tools/IDE to deal with.
tactics Thu 11 Mar 2010
For lookups, tools could just delegate to a live env chain.
I agree grouping multiple scripts into a single pod would be a good idea if we want to do hot loading.
katox Thu 11 Mar 2010
I like the idea of no import statements - all dependencies handled via pods and
Env
, even in scripts.I can't think of a case in Java when I really wanted to write some imports. I declare all dependencies for each subsystem in maven anyway why should I bother again? The only outcome is probably that I have to resolve import conflicts (among commiters) when we change some generic implementation to our own flavour.
Any decent tooling should be able to display pod origin of any class. I don't see a way how could I judge directly if
Event
class comes fromdom
orfwt
- could be either of them - there is no import of each class. It is the same information as I can get from declared pod dependencies.using as
is comfortable but as in Java I could live as well with fully qualified names for such rare cases.jodastephen Fri 12 Mar 2010
No imports sounds appealing. I think you'd have to deal with the ambiguities though. Maybe it as imports setup on a per pod basis.
tcolar Fri 12 Mar 2010
I don't think I like No import for several reasons:
tactics Fri 12 Mar 2010
Scripting languages don't have a build process and don't have any centralized way of naming dependencies. If they don't declare it per-file, they can't declare it at all.
However, Fantom lives in a weird limbo space. It wants to be both a compiled language AND a scripting language. (I really like that aspect of the language).
Are you saying
Notepad++
andvim
aren't decent tools? :) (Not to mention Andy and Brian have said they aren't heavy IDE users).Plus, there are lots of times when you're browsing code in a way that an IDE can't help you. For example, browsing BitBucket.
brian Fri 12 Mar 2010
I lean towards keeping using statements as they are for inter-pod imports. It isn't all that onerous even though its a bit anti-DRY with the pod depends. But has the benefit of being a little more explicit in readability of how source file's namespace is being managed (in case you copy files or change your pod dependencies).
However its a different issue if multiple script files should be bundled into one logical pod, or treated as individual pods. I'm really digging the idea of bundling them into one pod (in which case there is no using statement b/c they all live in the same pod).
I think we if also say that a "script pod" can't depend on other script pods, then we have a clean mechanism for auto-reload whenever one file changes in a script bundle. Not sure how that might effect common situations where you want a chrome.fan script to be used by multiple page scripts:
Ideally you'd want pod
page1
to be composed of "chrome.fan" and "page1.fan". Thepage2
pod would be composed of "chrome.fan" and "page2.fan". Then:Not sure how different scripting languages like Ruby work, but they are monkey patched at runtime so its a lot simpler to handle than JVM classes.
katox Fri 12 Mar 2010
@tactics
Well... I supplied vim syntax rules and still use it as my primary working setup so clearly this is the time when Matlock would object ;).
I can ask again how
using fwt
andusing dom
would help if you used anEvent
. In Java - OK - there will probably be an import of this class. But in Fantom there are only two lines (at best) - it could come from any those or it could be implicit.If there was nothing you'd have to go to pod declaration but so what? If you don't know any context of the source file it is not your biggest problem anyway (what about classes defined in other pod files etc). If you do know the context you probably know it comes from
fwt
ordom
pod (previous experience).I understand your fear - I am really not comfortable with a bunch of
import static
statements in Java - but in Fantomusing
statement is actually a quite different beast.@tcolar
You can look it up in your pod dependencies. The information is the very same (unlike in Java where you can find individual classes).
using fwt
is helpful in what regard exactly?I don't see it right now. I would appreciante the debate to reveal it, honestly. BTW Newspeak doesn't use them - G. Bracha left it to tooling.
KevinKelley Sat 13 Mar 2010
There's something to be said for limiting the universe of discourse... A larger project will depend on a quite a few pods. But each source file should not. Growing lists of
usings
is a good indication that you're losing cohesion in your design.Being able to narrow your focus to a particular "thing", whatever that thing may be, helps you to understand it and make it right.
using fwt
tells me that this is user-interface. Narrowing it down further byusing
the individual classes you need, is even better.Better tooling mitigates; with completion and context-popups you end up wanting usings to be auto-managed. But I don't think they should go away... the best tool will probably always be a brain and an editor.
andy Sat 13 Mar 2010
So how do you define which scripts get included? We would need something flexible enough to define pulling in scripts from a database or arbitrary directories - but also something that works out-of-the-box using all scripts in the current directory maybe.
Maybe that is just the default Env - and you can plug in a more complex Env for the database issue.
Either way this (single script pod) seems like a good middle ground between scripting and pods. We make scripting flexible enough to span multiple files (which we need) - but don't go overboard when we have a formal way to tackle that in pods.