This is a proposal for a bit of refactoring to the web and webapp APIs. First I want to recap our original philosophy on these APIs:

web: this pod is designed to be the standard API for integrating into web servers. We have wisp which is our simple Fantom server, but we also need to integrate into other J2EE platforms, maybe IIS, etc. So we want a standard Fantom API for plugging composable web apps into any server. But we want to stay as low level as possible so as to not dictate a given web framework; just make sure that we can easily compose a web-site using multiple independent web-apps which might use different frameworks.

webapp: the original vision of webapp was to build out one web framework which sat on top of web. Today this is a collection of reusable WebSteps for building out a web pipeline and Widget which is a way to compose chunks of HTML generation together with AJAX callbacks.

Proposal 1: remove webapp

There are a couple problems with webapp. The most important is that we aren't using it ourselves (we aren't dogfooding it). Originally all of SkyFoundry's products were going to be based on webapp which would prove out the framework in production. But now SkyFoundry is exclusively building everything with JavaScript-FWT in the browser talking back to the server with a REST API. Secondly, it is probably a bit too early for us to be trying to standardize one web framework bundled with the core Fantom distro. Rather I think our first job is to just standardize the foundational web layer.

So first proposal is to remove webapp from the core Fantom distro. We'll move webapp to the contrib repo if anyone wants to keep using it (no guarantees on maintaining it though). LogStep will be moved to wisp since that is sort of server specific functionality.

Proposal 2: WebApp

The other issue we've identified is that we really haven't done a good job yet of ensuring that I can write a webapp like a blogging engine and make it super easy to drop into a Fantom enabled web server. Andy and I had a long brainstorming session on this topic this morning, and decided we really should introduce a new concept which we'll call WebApp (not sure that name is perfect, so other suggestions welcome).

WebApp would bundle up a reusable chunk of web functionality which can be mounted into a server's namespace. After a long debate we decided that WebApp should be a separate concept from Weblet. Weblets are mutable chunks of code designed to service HTTP requests, where as WebApps are immutable packages designed to plugin into a web server's namespace. Most likely a WebApp would delegate to a Weblet.

The proposal is as follows:

• remove WebService
• remove WebStep
• add WebApp, FileWebApp, and RouteWebApp
• add WebReq.app, appUri

The new APIs:

** used to mount web functionality into a server's namespace
const abstract class WebApp
{
   abstract Void service(WebReq, WebRes)
}

** used to publish static files in a server's namespace
const class FileWebApp : WebApp
{
   new make(|This| f)
   const File file
}

** used to delegate URIs to sub-WebApps
const class RouteWebApp : WebApp
{
   new make(|This| f)
   const Str:WebApp[] routes
   override Void service(WebReq, WebRes)
   {
     // resolve route, update WebReq.app/appUri, 
     // delegate to route's service method
   }
}

class WebReq
{
  ...
  WebApp app
  Uri appUri // always ends in slash
}

The basic idea is to package up Fantom web functionality into WebApps and make it deployable anywhere into the server's URI space. Web code can use WebReq.appUri to figure out where it is living in the server if it needs to generate hyperlinks.

By getting rid of WebService and WebStep, we are getting rid of prescribing how WebApps actually get bound into a web server's namespace. We leave that up the server implementation. The way wisp will work is that it takes one top level WebApp which is responsible for servicing requests. An example boot script might look like this:

WispService
{
  app = RouteWebApp
  {
    routes = 
    [
      "index":  FileWebApp { file = scriptDir + `pub/index.html` },
      "docs":   FileWebApp { file = scriptDir + `pub/docs/` },
      "blog":   AcmeBlogWebApp { ... },
      "stats":  LogStatsWebApp { ... },
    ]
  }
}

But another server like Glassfish might plug WebApps in as a WAR file or something.

Another issue which we will need to address is authentication. That is a cross cutting concern that has to be addressed in the web API itself if we wish to make WebApps composable. But, I don't have a proposal for that yet. I want to get this change in place first.

Thoughts?

I'm all for dropping the current webApp. Using it has been pretty awkward. (Yet despite that, Fantom still makes it fun and easy).

I don't see any problems with the proposed changes. It shouldn't be very hard at all to emulate the old webApp pipeline: you just resolve the incoming Uri on a UriSpace, figure out which View that object's type uses, and service it. (I never understood or used the FindChromeStep).

Every web page needs a blog, or forums, or a wiki, and installing them is always a hassle. Every one has its own unique setup instructions, and you have to make sure this or that is deleted or renamed after running it. I like the idea that you can just create a new WebApp and hook it into your server.

One question though. How would Wisp decide where the WebApps will be mounted? How does Wisp know to call routes["index"].service instead of blog["index"].service? The routes are marked by keys, but do they also indicate the base URL? (Or not?)

Also, when you say authentication, what kind of authentication do you mean? Do you mean application-level username identities?

Either way, it sounds good. The design of the web application I've been working on is still pretty fluid. I'm still feeling out what works and what doesn't in Fantom and the web libraries. Once we have some implementation of these changes, I can see how tasty the new dog food is ;-)

I'm all for dropping the current webApp. Using it has been pretty awkward. (Yet despite that, Fantom still makes it fun and easy).

The current API is pretty simple and flexible, but it definitely has some problems we'll be solving. The biggest is that using the UriSpace as the default mapping for web resources is mighty convenient, but not easily secured. I also think that a single top level list of WebSteps isn't quite flexible enough, this new design will allow us to structure things more like a tree structure. For example plugging in new functionality can be a WebApp that wraps one or more other WebApps.

One question though. How would Wisp decide where the WebApps will be mounted?

Well in Wisp we own the whole URI space so we can assume the root WebApp is /. Then as in my example if I mount a RouteWebApp into the root, then that defines the first levels in the path. Let's take a more complicated example:

WispService {

app = RouteWebApp {
  routes =
  [
    "alpha": AlphaWebApp {}
    "beta": RouteWebApp
    {
      routes = ["gamma":GammaWebApp{}, "delta":DeltaWebApp{}]
    }
  ] 
} }

/alpha/*       =>  AlphaWebApp
/beta/gamma/*  => GammaWebApp
/beta/delta/*  => DeltaWebApp

Does that make sense? In practice you probably configure your top level apps in your boot script. But you might have complex WebApps which themselves delegate to sub-apps.

Also, when you say authentication, what kind of authentication do you mean? Do you mean application-level username identities?

We don't want to define any more than we absolutely have to. But at the very least all WebApps need to be able to check if a user has been authenticated, and if not know where to redirect them. We probably also want some general purpose name/value pair API to access user information without forcing any fixed user API.

Then as in my example if I mount a RouteWebApp into the root, then that defines the first levels in the path.

Ok. I just wasn't sure if the keys in the Str:WebApp corresponded to URLs, because they were Strs and not Uris. But it works as I thought it would.

We don't want to define any more than we absolutely have to. But at the very least all WebApps need to be able to check if a user has been authenticated, and if not know where to redirect them.

I understand that you're speaking in general terms and you don't want to pin anything unnecessary down. I'm just not sure which level of abstraction you are aiming for. Is what you mean going to play a role similar to Apache's htaccess files? Is the authentication there to restrict access to the UriSpace? I guess I'm curious what kind of scenarios you have in mind for authentication.

It looks good, though. I'm excited for it. Let me know if you want any help with it, since I'll be making heavy use of it.

I think you should also consider virtual hosts and protocols (HTTP vs HTTPS) while routing requests.

Is the authentication there to restrict access to the UriSpace? I guess I'm curious what kind of scenarios you have in mind for authentication.

I think the most basic thing is just an API to:

1. check if a user is currently authenticated and logged in for the request
2. ability for WebApp to redirect if not logged in (without knowing that URI ahead of time)
3. ability for WebApp to retrieve some name/value pairs from "user" record which hosting server would have to provide

I think you should also consider virtual hosts and protocols (HTTP vs HTTPS) while routing requests.

Not sure we'll build a class in the core web module for that, but it would be quite easy to build yourself. But I guess if this was a common request, it might be nice to add to standard library.

I really like this proposal of being able to create different WebApp to handle different routes...

But what will be great is to not have to defines route in the code but outside of the code or through a Facet on the pod...

I have in mind the following scenario: Wisp will just be a web server that route request to the right pod/WebApp... By default, wisp will respond 404 service not available... Then I drop a new pod in the well known directory, wisp detect it and load it -> either there is a facet indicating the route for it or external fansym file, or anything else... But then wisp knows that it can return a response for this specific rout by forwarding request to the new pod...

What do you think of this scenario... Wisp will be a kind of apache web server for fan pods.

But what will be great is to not have to defines route in the code but outside of the code or through a Facet on the pod

That is not far off from the way it works today. And it would be pretty easy to build a web framework that did that. But the major problem with that approach is security. Enabling a chunk of web functionality just because a pod is in your repo introduces all sorts of security risks. So that is why this model leans more towards excluded by default, with some explicit config needed to include a chunk of functionality. Seems a lot safer, and not too much extra hassle.

Although one cool way to approach the problem would be have a WebApp that searched the typedb to find other WebApps, and gave you a nice web page to check off the ones you wanted to enable.

On another related topic, I'm thinking WebApp is probably not the right term. What do you guys think of the term WebMod for a web module? That seems a little more generic, and also jives with ApacheMods.

Renamed from Proposal: Web APIs to New WebMod API

Promoted to ticket #836 and assigned to brian

Ticket resolved in 1.0.48

I have completed these changes and pushed them to hg. Overall I am very pleased with the new design - it is definitely more flexible than the previous design and should make for easier web module composition.

If you are working from hg tip, then the best thing is to go thru the updated docs and examples.

High level things to note...

Weblet is now a mixin which can mixed into mutable or const classes. I also renamed Weblet.service to onService to be consistent with other callbacks.

I decided to use the term WebMod instead of WebApp. WebMod is really just a special const Weblet with two additional lifecycle methods for onStart and onStop. Really what this change does is make how you configure your pipeline and routing more flexibile. The old pipeline model is still available via PipelineMod, but now you aren't required to use it - it is just one option in the new webmod libary.

This new model is also more flexible for dropping Fantom code into other web servers since now we don't assume Fantom owns the entire URI namespace. The new WebReq.mod, modBase, and modRel methods provide the the flexibility to develop a module which can be dropped anywhere into the URI namespace.

All the old code which was removed is available in the contrib repo (it doesn't compile, but feel free to pillar classes as you need them for your own projects).

Change Log:

• Weblet now mixin
• Weblet.service renamed onService
• WebMod added
• WebService removed
• WebStep removed
• UserAgent removed
• WebReq.service, resource, userAgent removed
• WebReq.mod, modBase, modRel added
• WebRes.service removed
• examples/web refactored
• removed @webView and @webViewPriority facets
• webapp removed
• webmod added
• WebSession implementation moved from web into wisp