#934 Uri Namespace

brian Tue 26 Jan 2010

This an updated proposal for #720.

Removal of UriSpace The UriSpace class was originally visioned as a core aspect of concurrency, web framework, and plugging resources into URIs. But its original significance has been largely wiped out by a number of factors:

  • easy of plugging in new top level schemes
  • focus on actor based concurrency
  • dismantling of original web framework into webmod design

The proposal is to remove the UriSpace API completely, and repurpose the "fan:" scheme for pods, types, and slots.

Fan Scheme Within a pod we have developed an ad hoc naming system for "pod::Type.slot' for types and slots. Then for resources we use something like fan:/sys/pod/icons/x16.png. Ideally we should unify this into a consistent format captured by URI. Also this is a good opportunity to unify namespace syntax with the syntax used in source code (which uses # instead of . as separator between type and slot).

Proposal is define the "fan:" scheme format as follows:

fan:pod            =>  resolves to Pod
fan:pod::Type      =>  resolves to Type
fan:pod::Type#slot =>  resolves to Slot
fan:pod/path/file  =>  resolves to Pod.files resource file

The following methods are changed to use an Uri:

Pod.name    =>  Pod.uri
Type.qname  =>  Type.uri
Slot.qname  =>  Slot.uri

All reflection APIs will use the new URI syntax (including Type#slot instead of Type.slot)

Scripts and FFI Scripts will use a pod "name" which is the URI used to resolve to a File. Any URI is legal as long as it doesn't contain unbalanced []. It is embedded into the "fan:" scheme as follows:

fan:[file:utils.fan]
fan:[file:../shared/script.fan]::SomeClass#someSlot

Likewise FFI plugins will expose a foreign namespace to Fantom via a URI scheme:

fan:[java:java.util]
fan:[java:javax.swing]::JFrame
fan:[dotnet:System]

Using Syntax Existing rules will apply except that the grammar for using statement will be slightly modified to allow URI literal instead of pod identifier:

using [java] javax.swing      =>  using `java:javax.swing`
using [java] java.util::Date  =>  using `java:java.util`::Date

You will also now be able to resolve other script files:

using `includeScript.fan`
using `myloader:test`

Qualified type names will continue to be limited to identifier::Type. If you need to use a qualified type from a non-standard pod then you will need to use an using or using as statement.

brian Tue 26 Jan 2010

Promoted to ticket #934 and assigned to brian

tactics Tue 26 Jan 2010

I don't have any qualms with using Uris to identify pods, types, and slots. I think it makes the using syntax more uniform, which is nice.

However, I don't like the way of mapping these units to Uris as they are laid out here. It's easy to see if you start up fasnh and evaluate:

fansh> `fan:pod::Type`.scheme
fan:pod:
fansh> `fan:[file:../shared/script.fan]::SomeClass#someSlot`.path
[[file:.., shared, script.fan]::SomeClass]

Even in the most basic cases, you have these weird quirks due to the URI syntax.

For FFI, we could potentially use an alternate scheme name. Borrowing the idea from subversion over secure shell (whose scheme is svn+ssh://...), we could use java+fan://... and dotnet+fan://.... A different scheme makes sense, because the FFI pods are resolved in a totally separate way than native Fan pods. Just like how you can't pull certain secure websites over regular HTTP, you can't pull regular pods over java+fan.

For scripts, the embedded file in fan scheme is massively awkward. If you are referencing a file in another directory, the slashes are unavoidably misinterpreted by the URI. We could URL encode the file path, but it would look pretty ugly. I think a better way to handle this situation is needed, but I'm not sure what.

Lastly, I think resource files should probably get their own scheme, too. Everything else proposed in the fan scheme deals with the type system except for this. Something like res:/icons/x16.png or fanr:/icons/x16.png (fanr for "Fan resource", like https is for "http secure").

andy Tue 26 Jan 2010

Even in the most basic cases, you have these weird quirks due to the URI syntax.

There is nothing about Uri's that say it must be a file path, so I wouldn't get too hung up on alot of the methods that intended for file/http Uris. We want to use the same syntax here as for type literals, so this is the best tradeoff I think.

If you are referencing a file in another directory, the slashes are unavoidably misinterpreted by the URI.

The brackets delimit the internal Uri here, so there should be no problem, unless you have a bracket in the file path, which you would have to escape.

Lastly, I think resource files should probably get their own scheme, too

One of the major goals here is to unify the pod structure under a consistent naming system, so I think its both a requirement and a simplification that they both use the same scheme.

brian Tue 26 Jan 2010

The fact that fan:pod::Type parses incorrectly is a bug.

Using double colon is definitely non-ideal; Andy and I had a long debate about what symbols to use which would provide consistency b/c URIs, reflection, and source code literals. In the end the path of least resistance seems to be to keep things pretty much the same, just switch . to #. What I want is that a consistent syntax is used across the board as the pod/type separator and the type/slot separator.

tactics Tue 26 Jan 2010

The fact that fan:pod::Type parses incorrectly is a bug.

I figured it might be after looking at the URI spec.

There is nothing about Uri's that say it must be a file path

Maybe not, but it's so common for protocols to use path names that the average user is expecting it. I think it would be very weird for a new user who came across Uri parseTime := `fan:[file:../shared/Utils.fan]::Utils#parseTime` in someone else's code. They'd probably think something like, "the backticks mean it's a Uri, but I've never seen a URI that looks like that!"

That's just my feelings on it. I'd like to see Uris used for reflection, but it seems like it might be hard to get a syntax that doesn't look awkward. I'd like to see what other people think.

brian Tue 26 Jan 2010

but it seems like it might be hard to get a syntax that doesn't look awkward

I think the whole point is that the URI syntax isn't much different than what we already do today:

Slot.find("web::Cookie.age")      // today with Str
Slot.find(`web::Cookie#age`)      // proposed with implied fan: scheme
Slot.find(`fan:web::Cookie#age`)  // proposed with explicit fan: scheme

Type.find("[java]javax.swing::JButton")   // today with Str
Type.find(`[java:javax.swing]::JButton`)  // proposed

jodastephen Thu 28 Jan 2010

Some thoughts:

  • Are we defining a URL or a URN?
  • Is this a Fantom virtual filing system?
  • Should we allow for hierarchical pod names? (this has been a repeating request)
  • The URIs being proposed look pretty ugly, and are rather "in your face" in the using definition.

I was also experimenting with making the URI look more "normal":

fan:web                     using web
fan:web/Cookie              using web/Cookie

fan://my-repo/web           using //my-repo/web
fan://my-repo/web/Cookie    using //my-repo/web/Cookie

fan://java/java.util        using //java/java.util
fan://java/java.util/Date   using //java/java.util/Date

fan:/sys/pod/icons/x16.png

Some of this comes down to a question of "why do we need a distinct symbol to separate the pod from the type" - currently we use ::. (Perhaps because it allows easy pod references in the main code, but is that actually needed with using...as...?)

Another aspect is to ask if the repo is like a server, and this could lead into a hierarchical path (which is much more eye-friendly).

I also like the fan+java URI scheme as an alternative:

fan+java://repo/java.util        using java://repo/java.util

brian Thu 28 Jan 2010

Are we defining a URL or a URN?

Fan URIs are both URN and URLs:

  • Normal pods are really URNs with a name which delegates to the Env for resolution
  • Script files are most likely URLs that explicitly resolve to a File which can be compiled into a pod
  • FFI are virtual namespaces mapped into Fantom's namespace

Remember this feature will tightly tie into Env as the delegation mechanism used to resolve a pod URI into an actual pod.

Another aspect is to ask if the repo is like a server, and this could lead into a hierarchical path (which is much more eye-friendly).

I think this is a mistake to put repository information into the pod identification. Pod names should be identifiers, not locations. It should be an external delegation model which figures out how to turn pod identifiers into actual files. How a pod is loaded from repositories is a deployment concern, not a source code concern.

I am not sure why there is a desire to have fan URIs looking like file paths. Lots of naming systems don't use slashes (email, phone numbers, URNs, programming languages, etc). Since we start with the syntax in the language, it seems natural to extend that to the URI. Everybody flamed PHP for using slash as their namespace separator :-)

Just to keep things on track, let's review my perspective of the requirements:

  1. pods, types, slots, and other things inside a pod are named entities
  2. in Fantom everything named should be named with a URI
  3. we want a consistent separator symbol b/w pod and type (today it is ::)
  4. we want a consistent separator symbol b/w type and slot (today it is #)
  5. we need to have flexibility in pod identifier to support any URI to a script
  6. we need to have flexibility in pod identifier to support FFI namespaces

Once you start really digging into the problem, the root issue will always come back around to number 5 and the fact that URIs with different schemes can't be composed (the #1 flaw in URIs in my opinion).

brian Fri 29 Jan 2010

Andy and I did a bit more brainstorming on this. Basic problem as I enumerated above is to provide flexibility in:

  1. how pods are identified (normal identifiers, file URIs, FFI namespaces)
  2. define namespace within a pod (types, slots, resources)

Here is our latest working proposal:

fan://pod/Type#slot
fan://pod/res/file.txt

Examples:

fan://sys                 => resolves to Pod
fan://sys/Str             => resolves Type Str#
fan://sys/Str#each        => resolves Method Str#each
fan://sys/locale/en.props => resolves to File

Couple notes:

  • developers are responsible for organizing the "path space" of their pods with types, resource directories, and eventually maybe pluggable services
  • use of / as pod/type separator replaces ::
  • use of # as type/slot separator replaces .

Normal URI parsing will give you the pod/type/slot parts:

`fan://sys/Str#each`.auth   =>  "sys"
`fan://sys/Str#slice`.name  =>  "Str"
`fan://sys/Str#slice`.frag  =>  "slice"

As the "auth" component of a URI, there is good flexiblity in the syntax of the pod. Anything without slashes can just be used, or we'll still use [] for embedding other URIs which have slashes (like IPv6 hostnames in a http URI):

`fan://java:javax.swing/JButton`
`fan://[file:../dir/script.fan]`

Also to simplify the language and make things cleaner, we decide to remove support for qualified type name within normal code. Qualification will be strictly an aspect of the using statement. If you need qualification to avoid naming collisions, then you can use using as.

How does that suit everybody?

tactics Sat 30 Jan 2010

This sounds like a fair compromise.

qualidafial Sat 30 Jan 2010

Could you give some examples of using and using ... as statements?

brian Sat 30 Jan 2010

Could you give some examples of using and using ... as statements?

Not sure actually how this should look. I think the cleanest thing is assume that the "fan://" is always inferred:

using web
using web/Cookie
using web/Cookie as Ck
using java:java.sql
using java:java.sql/Date
using java:java.sql/Date as SqlDate
using includes.fan
using ../mydir/includes.fan

andy Sat 30 Jan 2010

I think you need to use ` in at least the last two examples - we require them everywhere else, so lets not make this a special case. Then you I think you could use the rule, if no ` found, then assume fan:// prefix.

brian Sat 30 Jan 2010

The problem with requiring them is that this

using `file:includes.fan`

Actually means

`fan://[file:includes.fan]`

And I don't think we want to force people to always do:

using `fan://java:javax.swing`
using `fan://[file:../includes.fan]`

So it seems like for consistency it is better to always leave them off and just let compiler turn it into proper "fan://pod" URI. Otherwise I have to remember when I can and cannot leave off them off.

andy Sat 30 Jan 2010

I agree we don't want to require people to fully specify the pod URI. Thats why I suggested only requiring for file paths (personally I see that as a req). But a simple set of rules:

  1. No backticks -> simple pod name, no brackets
  2. Backtick, no scheme at all -> insert into bracets with file:
  3. Backtick, scheme != fan -> insert into brackets
  4. Backtick, has fan scheme -> already fully qualified

So using those rules:

using web
using web/Cookie as Ck
using java:java.sql
using java:java.sql/Date as SqlDate
using `includes.fan`
using `../mydir/includes.fan`
using `http://foo.com/foo`

andy Sat 30 Jan 2010

In practice, that a pretty simple thing to remember:

  1. simple pod names, or FFI - just use name as indentifier
  2. import a script - use normal Uri like I would anywhere else

jodastephen Sat 30 Jan 2010

This new proposal looks great.

I agree with Andy's modification. Essentially, a using line without URI literal is normal fan stuff, while with a URI literal is advanced, where file is the default protocol. I'd like to avoid Andy's rule #4 though, however it just might be necessary.

brian Fri 7 May 2010

Ticket cancelled

Going to just leave the FFI syntax completely alone.

If we do support the file system directly as a FFI, then it will just be [file] path. Although there are other ideas floating around, and I don't think we need to make any final decision for that in the core immediately.

Login or Signup to reply.