You really need to pop into London Town so you can join me and Justin discussing this kind of thing These are the kind of posts I was hoping to see in response to Julian’s meme-starting antics. I wish I’d posted something more serious than a drunken MLK babble ..

Jon

You are right about authoring though, there is absolutely no reason not to throw lots of dedicated CPU time at making the authoring experience easier may as well shift the work to the client side if possible, make those multicore laptops do something. As you say this is a low traffic environment.

I notice Google Wave tries to do shared editing without merge conflicts (atomic updates only, always ordered by the server, no attempt to support disconnected operation as far as I can see), but I think you do need disconnected operation and interfaces for resolving merges.

That said, while I agree that links should be represented as “dumb” URIs to the author, that representation isn’t sufficient to be able to support the kinds of operations I mentioned above (specifically efficient bi-directional traversal and queryability). For those, the CMS needs to have a persistence model that directly models the graph that those URIs represent – hence the mention of Neo4J, (although it’s by no means the only graph database out there – CODASYL has been around for quite a while!)

As you say, sharding is a thorny problem for interrelated data structures such as graphs – there’s not necessarily a clean way to split the content graph into shards without losing the ability to efficiently traverse / query the entire graph. But in my experience sharding is primarily a performance optimisation (ie. is of most use in the the delivery tier where internet scale is important), while many of the operations mentioned above are primarily in support of authoring activities (where traffic is minuscule, by internet standards). In other words these two limitations (can’t have both sharding and efficient graph traversal / query) don’t have to intersect – the CMS can optimise the authoring environment for efficient full-graph operations (at the cost of not being able to shard), while optimising the other way around for the delivery tier.

In fact I think replication is more important in authoring than sharding, as that supports the requirement for disconnected and/or geographically distributed authoring (think multi-national corporation with authors on every continent).

Actually I think we are in agreement here too. There are three ways you need to use links, one the link from the authors point of view (I want to link here, so I reference it with a URL or some sort of internal reference), two internally is it strictly a foreign key or not, and three what can I do with (valid) links, all the things you describe, which are as you say key things a CMS needs to give you.

I think that the problems with strict foreign keys/internal references are two big (no sharding, how do you link to a page you have not written yet, all sorts of issues), so you need URI based links (although those might not be deployment URLs, but they are resource identifiers more generally) to name things, but the state of being broken is allowable, but there are tools to help the user manage this brokenness, in order to fix it.

Internally you do need access to the full graoh of (non broken!) links though in order to build your IA.

Links need to be first class objects too, as they need to be permissioned and versioned (eg you cannot see in your graph a private link someone has made).