I've been having some email discussions with @andrey regarding their efforts to run Fantom on OSGi for Eclipse. Now I personally don't particularly like OSGi, however I definitely want to make sure Fantom runs on OSGi really well for those developers who do like it (or need it). Since Fantom already has nice a modular design, it should be a fairly nature fit to map Fantom pods to OSGi bundles.

Andrey has indicated a couple key issues required to make Fantom run well in an OSGi environment (these are his quotes):

• More flexible repo structure, and I'd like to propose a kind of fan.bootpodpath property, similar to classpath this is a list of absolute paths to pods to include in boot repo. This feature is essential for us both to run Fantom VM inside OSGi container and to our IDE work (projects/workspace structure very likely would not fit repo directory layout, and we'd like to run/debug pods under development without deploying them to VM install dir). bootpodpath handle both tasks well.
• Classloading-per-pod approach... Besides our OSGi needs, and first of all, I'm strictly believe this is right and essential need for Fantom-as-a-Platform theme. All the good Java platforms I know following classloader-per-unit-of-deploy pattern, and Fantom can benefit in future if utilize such mechanism in many places like pods un(re)loading, flexible management of multiple versions of some pod and pod dependences in general, discarding run-time compiled code, etc

I am not sure I'm ready to tackle the first issue. Eventually we need to seriously reconsider repos vs pods, and how they relate to alternate runtimes (like OSGi, Jigsaw, J2EE) and how they will work with cloud based repositories.

For this discussion I'd like to focus on the classloading issue because I think it is easier to get our head around and will require stubbing out some pluggable environment hooks in the JVM runtime. Remember OSGi is just one potential kind of container - hopefully after this exercise is complete it should be easy to run Fantom in any type of container.

The key question is should Fantom standardize on a classloader per pod? At Tridium I used this design for Niagara's module system and it worked well enough. But for Fan I started off with the simplest thing that work - a single classloader which traps anything with "fan.{pod}.{type}" and emits fcode to bytecode on the fly. I don't have any strong opinions on moving to classloader per pod. Doesn't seem like there are any big negatives, and there are definitely GC benefits over the current design. It seems to me like the easiest thing is to just standardize on a classloader per pod if that is what OSGi needs. As long as that doesn't make J2EE containers more difficult, then does everyone agree on that design?

EDIT: one thing this feature might dovetail into is using URIs for Pods to enhance script includes (see #720).

Note that the only way to reload types in .NET is at the assembly level (pods match up 1:1 to assemblies). So a move to a ClassLoader-per-pod would seem to capture that model across (current) platforms.

Even in Java just about every app server I've used, uses 1 classloader per application deployed (war) - with a common "master" classloader.

It just makes everything simpler and avoid name clashes and the likes.

I actually think it could make "deploy a pod" to a JVM easier.

ho ...but never mind. In Fantom their is no "package" like in Java, so the smallest unit to "organize" your code is the pod.

So now one classloader per pod would mean your entire webapp would have to be in one pod- then no, for any kind of medium to large size app that would be ugly.

I think really Fantom needs a layer above the pod when it comes to deployment, the equivalent of the webapp in java. Basically a bunch of pod and maybe a minimal config file if absolutely needed (~web.xml), and THIS unit of deployment gets it's own classloader.

That's what would make sense to me.

Wasn't Repo exactly what you talk about? It ain't organizational but it it could take a bunch of pods and config file and is the rough equivalent of WAR.

So now one classloader per pod would mean your entire webapp would have to be in one pod

That isn't right, is it? Classloaders can chain, right, so the webapp starts in its private classloader, uses several pods which each loads through its own classloader... as I understand, your classloader takes its permissions and so on from its parent loader.

If the framework unloads your webapp, it kills off the loader that loaded it and thereby any further ones it loaded. At least that's how I understood it to work.

So far as I know, there's not really a downside to classloader-per-pod, and there's some upsides like being able to unload/reload at the pod level if you need to for some reason.

Could certainly be framework issues I don't know about; but unless somebody knows better I think we need to be try it and see how it works.

You are right, they can chain. But if the pod classloader always delegate to the system classloader, then you can have issues where two applications deploy use the same pod, and you run in confusing issues

If duplicated classes in the classloader and what not, that's why having an "application" level classloader in between is useful (in Java at least).

Maybe that's what repos are first, I didn't think of them at a unit of deployment but then again haven't read much about it.

Anyway I don't know exactly what Brian & Andy plan are, personally I would like to see it possible to deploy a bunch of pods together as an application.

Also I think It it's possible it would be VERY cool if that deployment bundle could be dropped in a JVM and work as is, so if possible at all, i'd love to see that happen.

As a matter of fact here is Ted Neward's quote (twitter) shortly after trying out Fan:

The "pod" mechanism is a tad strange though--the pod can't be executed except within the Fan installation directory structure?

My opinion is that classloader-per-pod is good approach, and one of the benefits is handling different versions of the same pod in the same repo in a natural way. So, application can use two different third-party pods where each of these pods require its own version of the same pod. For cases when unit of deploy is bigger than pod (i.e. webapps), this unit of deploy can have its own classloader, as Kevin mentioned

I agree with tcolar and Ted Neward comment... The repo concept is too much of a developer concept ala maven - where to develop your pod(s), how they access common other libraries, resolving dependencies, ...

There are not in my mind a end user deployment unit at all and I think this concept is too absent from the Fantom distribution...

I would like to see something part of the build script that will create a unit of deployment based on your development repo, resources, dependencies, ... that can then easily be runned by a freshly installed Fantom distribution (perhaps a stripped down version of it - just for end user runtime) from anywhere, or better, directly run by the jvm or .net platform - but that will mean always distributing sys.jar in every unit of deployment.

Promoted to ticket #867 and assigned to brian

At this point repos are just a convenient way to organize code on the disk. I'd rather not start baking more semantics into repos since we already have as pods the fundamental unit of modularity.

Independent of other ways we might enhance deployment, I don't think I heard any nays on a classloader-per-pod. So I'll try to prototype this soon, probably move FanClassLoader into a new package like fanx.env and stub out a FanEnv class with the hooks to make bootstrap more pluggable (such as a classloader factory).

It's awesome to see such a movement in modularity area, as for non-OSGi containers, such as JEE, I believe classloader-per-pod would work really fine:

As I understand the main question for Fan is single classloader vs. many classloaders.

For Fantom embedding case (running within Java containers), which has many classloaders and own modular system, single Fantom classloader will not work, cause we'd need to flat environment's classloading: make all required (for FFI) classloaders in the environment available to FanClassLoader, which is

* hard (at least because good container will hide all the classloading internals, and that's fine); * or impossible in general case (just consider two different versions of the class with same FQN running in different web-apps). * also I guess such design would look just terrible

Of course we have an alternative to run complete Fan VM (starting from sys.jar) per web application, which will work well for single classloading approach, but this is heavyweight deployment. It could be fine for isolated apps (web-apps for JEE), but not a solution for containers which share data between modules (like Fantom itself).

On the other hand, classloader-per-pod looks very natural to Fantom, and besides potential benefits, which already mentioned (versioning, pod unloading), this approach should work well for N:1 (pods:units-of-deploy) scenarios like web-app built from N pods, @tcolar mentioned.

I would agree with @KevinKelley that there is no need to have a single pod per web-app, it's pretty much fine to have N pods per web-app. Fantom-J2EE bridge should be responsible for that N pods lifecycle in sync with web-app, and be able to delegate FFI classloading from all N pods to web-app's classloader properly (web-app classloader provided by environment).

So to me N:1 deployment do not very much different to 1:1 deployment. Moreover such web-apps may benefit from classloading-per-pod as well. For example someone may want to load/unload particular web-app pods or use pods as a plugins inside some particular web-app. But this looks more like a Repo issue/configuration.

So my assumption is that classloader-per-pod design is most flexible one and enough to support any deployment scenarios (except sub-pod deployment, which looks like a nonsense).

Kind Regards, Andrey

Makes sense.

I'm a bit late to the conversation, but could someone explain what this ticket means? I have the general flavor of what a classloader does. It's the thing which resolves fully qualified type names to their class file and loads it into memory. I know it can also do other fancy things. But (in simpler terms) what is the main idea behind having each pod load its own classes?

But (in simpler terms) what is the main idea behind having each pod load its own classes?

It is really just a better way to scope a pod. Classes are scoped within a classloader, and not GC'ed until their classloader can be GC'ed. So putting each pod into its own classloader allows unloading of pods or potentially running multiple versions of the same pod. But for practical purposes I think this change is mostly just to enable developers like Andrey to run Fantom on OSGi (and hopefully make it easier to run Fantom on J2EE).

In the simplified form, please consider classloader as a String->Class map, where key is a fully qualified name.

So having classloader per pod would let you

• keep different versions of java classes per JVM (of course should be loaded by different classloaders) -- versioning
• GC classes by removing references to classloader -- pod unloading

On other hand, and using that fancy classloading things you may have fine grained control on classloading behavior, like security, which was already mentioned on the thread, which can be efficiently done on per pod basis.

So if we assume that classloading for a pod is very much tied with that pod structure/definition, one to one pod-classloader association may simplify design drastically since you always know which pod is trying to load a class, and which classloader to use to emit, load, and unload Java code.

Ticket resolved in 1.0.50

The JVM runtime now uses a FanClassLoader per pod.

Classloading is delegated to two JVM-only methods on Env:

/**
 * Load the Java class representation of Pod constants.
 * Default implementation delegates to parent.
 */
public Class loadPodClass(Pod pod)

/**
 * Load the Java class representations of a Fantom type:
 *   - Fantom class => [class]
 *   - Fantom mixin => [interface, body class]
 *   - Fantom Err class => [class, val class]
 * Default implementation delegates to parent.
 */
public Class[] loadTypeClasses(ClassType t)

Alternate environments should be able to implement any classloading scheme they desire from these hooks. The BootEnv implements the standard scheme by looking in the pod files for precompiled Java classfiles or emitting the fcode to a classfile.