Alfresco Hybrid Sync

September 19th, 2012 by David Webster

It was back in February when I first started working on Hybrid Sync: helping get a demo together for an internal company event & it was from then that the momentum increased, along with the team size, and the last few month have seen a lot of effort from all across the engineering team to fix some “interesting” problems relating to document sync. The team have worked hard and I’m really pleased with the result.

The major use-case we were asked to address was Alfresco users in the enterprise wanting to collaborate with individuals outside their firewall using Alfresco in the cloud – but I can see countless other situations where being able to sync content off-site and into the cloud would be very useful. I think particularly that I’ll use the new quick share functionality to share content and use sync to ensure that external individuals always have access to the latest version.

I’ll let you find out more about the marketing yourself (see press release and comments from Paul Hampton and Kathleen Reidy), and instead I’ll look at how it works technically.

How does it work technically?

Accounts:
The first step in a sync is authenticating with Alfresco in the Cloud – this is done through the new Cloud Sync panel on a user’s profile page or through an inline prompt when first asking to set up a sync. In both instances, these details will be saved (in a new & secure credentials store) for future sync actions and are checked when you enter them (so you can’t save a bad password). In the future, we expect to replace this with an oAuth implementation.

You can sync to any account you have access to in the cloud. An on-premise user can only store the credentials for one cloud account, but a cloud account may have multiple on-premise users connecting to it. Any modifications made in the cloud will be pulled down as the on-premise user and any changes made on-premise will be pushed as the cloud user, so the UI at either end will only ever show the local user.

Sync Sets:
The next step is to select files for syncing, this is done through the on-premise Document Library (all sync actions are initiated on-premise). You can either multi-select files or choose to sync them individually through the appropriate Doc Lib actions. Under the covers, Alfresco has the concept of sync sets: these are groups of files that sync to a particular location. If you’ve selected multiple files then they will be grouped together as a single sync set, otherwise a sync set containing a single node will be created. A sync set is owned by the user that created it and will use that user’s cloud credentials to sync.

Pushing and pulling of changes is done on a sync set by sync set basis, but currently all sync management needs to be done on an individual file basis & sync sets themselves are managed by the system – Admin/User level sync set management features are planned for a future release. If you have synced a folder then any sync management actions (i.e. unsync) need to be done on that folder, and not it’s implicitly synced children.

Change Audit:
Once a node is added to a sync set, it will have a sync:syncSetMemberNode marker aspect applied to it & all changes to that node or sync set membership will trigger an entry in the audit log. The first entry will be the addition of that node to the sync set and this will trigger a node creation on the remote system. Periodically (every 10 seconds) this audit log is checked for changes and any sync changes found are sent to the remote server. On Alfresco in the cloud, changes to synced content are also stored in an audit log and are pulled by the local system every 60 seconds. All changes are aggregated together, meaning that only the final state of the node is pushed and versions are not created for interim state. However a version is explicitly created every time a change is pushed or pulled, even if it is only a property change and your local repo is set to not version on property changes.

Conflicts and Version handling:
Occasionally you will find that a node has changed in two places at the same time – there are lots of complicated was that could be used to mark and resolve conflicts, but we’ve taken the approach that we want to simplify this process. Being aware that the small poll period makes conflicts unlikely and that the major use case is to enable collaboration in the cloud, we have implemented a “Cloud wins” approach to conflict resolution. If a file changes at both ends simultaneously then the on-premise node will be versioned and will then be overwritten with the node from the cloud. To see the changes you can view the version history & revert or update as necessary.

What gets synced:
The node content and any properties from common content models (these can be found in sync-service-context.xml under propertiesToTrack config section). If the file is being synced as part of a folder sync, then the directory structure will also be synced (e.g. the structure in the cloud will remain the same as the structure on-premise), but if you’ve just synced the file individually, then just that node is synced and you can move it around without affecting the sync.

Unsyncing:
When you unsync, the sync:syncSetMemberNode aspect is removed and a record is added to the audit log (deleting a node on-premise also triggers this) – when the audit log gets queried for changes, this node removal is pushed to the cloud just as any other change and will optionally delete the version in the cloud.

Errors:
When a node cannot be synced then an error aspect gets applied. That error may be transient (e.g. comms failure with the cloud or authentication failure) or require a user interaction to solve (e.g. a name conflict in the sync folder or a permissions change means the target folder is no longer writable). In either of those cases, a marker aspect gets applied (sync:transientError or sync:failed respectively), the user is notified through Doc Lib indicators and banners and these aspects also affect how the audits work. When a transient error is hit, the audit logs continues to keep a record of changes and the system will automatically recover when it is fixed, but when a hard error is hit, the audit log clears current entries, stops recording changes and will only start again when the user manually requests a sync. A request sync triggers the full push of a node if the node has an error on it. If the push of that node fails again, then the error aspect will be reapplied.

For more information on setting up sync, see the documentation and FAQ.

That was a brief overview of some of the technical concepts that Alfresco developers and implementers may want to know about, let me know in the comments if you’d like a more in depth look at any part or if you’ve got any technical questions about this new feature.

Aloha Editor Extension Demo

February 14th, 2012 by David Webster

On Friday I was at the very first European jQuery conference. One of the tips I picked up there was about Aloha editor; they had a very good 30 minute intro to it & I thought it was worth exploring further – I also made other notes of the day. Yes, it was another iDay, and another tool based on jQuery (just like FullCalendar and Peerbind).

The Wiki is an area of Alfresco that seems popular with users, yet I really don’t like TinyMCE – the rich test editor that we use: from a maintenance point of view it can just suck time. With those two things in mind, I thought the best way to test it out was to create an extension that disabled the main editor interface and enabled the Aloha editor.

Having played around with it for a bit – I’m not sure that Aloha is ready for the prime time just yet (there were a couple of occasions it behaved a little bit funny and lost some formatting) – and the license counts it out for distribution with Alfresco anyway – but I liked it. It felt much more natural being able to edit content in-situ like that without needing to fire up a new window where you’re constrained by an input text area.

In his talk at the conference Haymo Meran showed that their research indicates that this editor would be 25% quicker to use than other rich text editors due to the UX improvements they’ve made, especially in reducing the number of actions a user needs to perform to complete a task. I can certainly see that those stats could be true – especially when you factor in a page load and refresh that aren’t needed. One area I didn’t get chance to look at was the built in repository browser, which is supposedly CMIS compatible – that would make it ideal for embedding or linking to content from elsewhere in Alfresco.

I created an extension (code on GitHub) – and jar here – that quite simply (in just a few lines of Javascript) invokes the Aloha editor, hides the normal edit link and then posts any modifications back to the server. This code is a proof of concept and shouldn’t be used for production use – I expect there to be a few issues with the version numbering, so some extra code would be needed to deal with that – and there’s no error handling. I think however that this demonstrates the flexibility of the new Share Extensibility system – with just a few lines of code and a small number of files, it’s possible to completely change the functionality of existing Share features.

I’ve got the extension descriptor that adds a new web script (with an empty template file). All the new web script does is add a few extra lines into the head on the wiki-page, calling in the Aloha library (from aloha-editor.org – you’ll probably want to host locally, but for ease of this example I just linked it) and an additional client side javascript file that does the magic.


Aloha.ready( function()
{
   var wiki = Alfresco.util.ComponentManager.findFirst("Alfresco.WikiPage"),
      $ = Aloha.jQuery;
   if (wiki.options.permissions.edit)
   {
      $("a.tabLabel[href$=edit]").next().andSelf().hide();
      var wikiContent = $("div.rich-content");
      wikiContent.aloha();
      Aloha.bind( 'aloha-editable-deactivated', function ( event, params )
      {
         // Save the results to Alfresco.
         var version = wiki.options.versions[wiki.options.versions.length -1];
         Alfresco.util.Ajax.jsonPut(
         {
            url: Alfresco.constants.PROXY_URI + "slingshot/wiki/page/" + Alfresco.constants.SITE + "/" + version.title,
            dataObj:
            {
               context: Alfresco.constants.URL_PAGECONTEXT + "site/" + Alfresco.constants.SITE + "/wiki-page?title=" + version.title,
               currentVersion: version.label,
               forceSave: true,
               page: "wiki-page",
               pagecontent: wikiContent[0].innerHTML,
               tags: wiki.options.tags
            }
         })
      });
   }
});

Hope that’s useful. Let me know in the comments what prototyping you’re playing with using Alfresco 4‘s extensibility framework.

David.

Alfresco example extension: Peerbind demo.

October 7th, 2011 by David Webster

I’ve just spent another iDay looking at combining Peerbind with Alfresco. Peerbind is a framework (written as a jQuery plugin) for allowing different instances of a page to communicate with each other. When I first read about it I thought it sounded very cool and it got me thinking about what something like that could do for Alfresco. Example use cases I’ve thought up for this are:

  • Real time notifications, so if you’re browsing Share, you’d get a live activity stream telling you what other Share users are doing; if you’re viewing a document and someone uploads a new version, you could get told about that and switch to the new version.
  • Online status, so you can see who else is using the site
  • Chat, so you can have a live discussion within Share, perhaps focused around a document that you’re reviewing.

What this demo achieves is a rudimentary chat client that appears on the bottom right of the document-details pages and a list of other users who are viewing that page. From this simple demo it’s quite easy to see how the functionality could be expended to include all the use cases above and more.

Document Chatroom Screenshot

This code (available on GitHub) is just me playing with an exciting new bit of tech: the code I’ve produced isn’t production ready, these features aren’t yet on the official road map or backlog, but it does serve as a good example of how agile the new extensibility work has made Alfresco 4. In literally a few lines of code I was able to take a brand new framework, plug it in and start innovating. David Draper’s blogs were a great starting point in how to get this extension done.

Set up the extension:

The first thing to do was create the Web Script that would load the HTML, CSS and JavaScript I’d write and load the external libraries (Peerbind and jQuery) as well as the extension XML using details I grabbed from SurfBug. I decided to extend the node-header in the document-details page. These are the files I needed:

Each of those files is pretty self explanatory (the .js one might not be, but is commented). I’ve got a jar file with them in that you can download – you’ll then need to follow the instructions in David Draper’s blog (drop in the Jar file, restart, enable module).

Chat Client:

The use case I looked at first was the chat client. Setting up a page based chat is very simple in Peerbind – they’ve got code on their homepage that does it; I’ve that code as a starting point. The premise is simple: on one end trigger a Peerbind event when the user hits enter inside the chat input box and on the other listen for the event. In both cases you’ll want to take the string from the event data and append it to a DOM node containing the conversation:

   /* Chat Window */

   // Cache chat root node and define local function.
   var $chat = $("#peerbindChat"),
      addChat = function Peerbind_addChat(msg)
      {
         // using append not prepend to put new messages below old messages, like other chat clients.
         $chat.find(".chats").append("
"+msg);
      };

   // The chat window starts off hidden, but clicking the title expands it all to show everything.
   $chat.find("h1").click(function()
   {
      $chat.find(".chatWindow").toggle("slow");
   })

   // the object with peer and local functions indicates that the callback has different methods depending on if the event
   // was triggered by the local client or a remote peer. It avoids the need for an "if (e.srcPeer)" statement.
   $chat.find("input").peerbind("change",
   {
      peer: function(e)
      {
         addChat(util.getUserName(e.srcPeer) + ": " + e.peerData);
      },
      local: function(e)
      {
         // TODO: This and other hardcoded strings should be internationalised.
         addChat("You: " + e.peerData);
         // empties the input field.
         $(this).val("");
      }
   });

If you’ve been paying attention, you’ll notice that instead of printing out the “e.srcPeer” string (as in Peerbind’s demo), I pass it to the function: “util.getUserName” – which brings us to the next part:

Online Status:

By default each client connected to the Peerbind server gets a unique id (srcPeer) which is sent on each request and response. I wanted these mapped to a username, so in my demo, I’ve got an “online” event that triggers an action on all visitors to that page. This action basically says “Hello, I’m a new visitor, and my name is:”. Everyone who is visiting the same page is listening for this event and when they receive it they do two things:

  1. remember the name and store is against the srcPeer id for later lookup (using util.getUserName) and
  2. send back a response (“onlineAck”) acknowledging the new client: “Hello, nice to meet you. I’m also on this page, my name is:”.

The response is targeted at the client that sent the initial online message (using their srcPeer id), so it isn’t broadcast to everyone and the clients who have already said hello to each other don’t end up getting duplicate introductions. The new visitor stores the username and srcPeer id for all the clients who welcome him. This is based on the “available” event example in the Peerbind documentation, except that in their example, they didn’t have the “onlineAck” event, so each client was only able to know about clients who arrive after they have and had no way to determine who was already there. My code displays everyone who is currently active on that page.

Here’s the code for triggering and listening for the online and onlineAck events:

   /* Online Status */

   // Trigger an online message & listen for responses.
   var addToOnlineList = function peerBind_addToOnlineList(id, userName)
      {
         $("#peerbindStatus .online ul").append('<li>' + userName + '</li>');
      },
      newClient = function peerBind_newClient(e, ack)
      {
         // events are triggered on both local and remote clients.
         // if there's a srcPeer identifier, then it's a remote client.
         if ( e.srcPeer )
         {
            // Remember the ID and store the username against it.
            util.setId(e.srcPeer, e.peerData);
            addToOnlineList(e.srcPeer, e.peerData);
            // send back an acknowledgement so they know we're online too, but don't send it back if we receive an ack.
            if (!ack)
            {
               $(document.body).peertrigger("onlineAck", Alfresco.constants.USERNAME, e.srcPeer);
            }
         }
      }

   // Set listeners for online actions:
   // - online is effectively a broadcast ping
   $(document.body).peerbind("online", function Peerbind_online(e)
   {
      newClient(e, false);
   });

   // - onlineAck is response received from the clients.
   $(document.body).peerbind("onlineAck", function Peerbind_onlineAck(e)
   {
      newClient(e, true);
   });

   // Tell everyone we've just joined and let them know our username.
   $(document.body).peertrigger("online", Alfresco.constants.USERNAME);

There’s not a huge amount more to it than that: the rest of the JavaScript is a slightly over complicated (for this use case) id map manager, but that was built with some future functionality in mind. If you unhide the #peerbindStatus div, you’ll see a list of who is viewing the page at the moment – this could easily be expanded into a more featured users information panel showing you their status, a link to their profile, their most recent action, etc.

Caveats:

As I said at the top – this isn’t intended as a production piece of code, it’s a proof of concept/excuse to play with something new/example of how easy share is to extend. Things I’d look at before deploying for real:

  • I wouldn’t want to rely upon Peerbind’s public server – I’d port/rewrite the server and host locally within Alfresco.
  • Online status should be domain (or site?) scoped rather than tied to the page. I’d create an ‘onpage’ subset of online folk to indicate who was available to chat with.
  • The offline trigger doesn’t work reliably (due to how Firefox handles the unload event) – I’d find a different way to do that, or create a timeout to remove users form online list.
  • Sometimes messages don’t get through (particularly if clients have been on the page but not active for a while). I think this is due to load issues on Peerbind’s server.
  • Styling and behaviours. The UI could do with tweaking (read: designing). The CSS needs fixing to get around iOS’s position:fixed bug.
  • i18n. I’ve hardcoded some strings for speed and to remove the need for an external .properties file. Obviously these strings should be internationalised.

That’s all for now. If you’ve got any ideas for how (or if) this could be used in Alfresco, or if there are other new exciting technologies you’d like to see explored in a similar fashion, let me know in the comments. Thanks, David.

Share Calendar Updates

September 26th, 2011 by David Webster

Some of you may have noticed a commit I made a couple of weeks ago. This completely replaces the way the Share calendar displays events. On the surface you’ll see minor visual tweaks, but underneath the code is brand new and hopefully they combine to give you a better user experience.

The Process:

One of the great things about working for Alfresco is the iDays we get, where we can go off on a tangent from our everyday work and explore tech that interests us. I’ve been keen to make Share’s calendar more useful, so I recently spent an iDay or two looking at the best way to improve it. I’m not one to reinvent the wheel and one of the biggest issues we’ve had is the logic to render events correctly in all the different views, so I was looking for a solution that would solve that problem for us. It wasn’t long before I settled on Adam Shaw’s excellent FullCalendar as the best of the breed – it manages the rendering in a robust way, does a sane job of exposing extension points and is written in a style that makes sense to me (important for on going maintenance).

A couple of weeks later, Share had a new calendar. Integration was fairly straight forward: mapping our event object, binding callback events and styling. I needed to make a couple of tweaks to the FullCalendar source code to increase our styling ability (adding classes so that all-day and multi-day timed events could be styled differently from single day timed events) and to enable it to fit our REST API (which requires dates in a slightly different format). With FullCalendar being open source, I’m hoping to get these enhancements pushed back into their source code so that other users can benefit from them.

Toolbar Tweaks:

Although there wasn’t much scope for modifying the overall look and feel of the calendar (this upgrade was mainly about improving the reliability of the existing calendar), Linton (our UX guru) and I did take the opportunity to improve the toolbar a little based on the results of a small UX testing session. This included reordering the buttons so that the add event one is more prominent, grouping the navigation elements and adding a Work hours toggle button to enable people to see their events in the Day and Week views without having to scroll past lots of blank time (1am – 6am isn’t a very common time for events, so why show it first?).

Image of the toolbar

Event Info Dialogue:

The popup dialogue box where you view or add event information was a constant source of bugs, so I’ve refactored it as part of this upgrade and hopefully now it’ll work as expected in a lot more situations.

Config Options:

The best part of the calendar replacement for Share admins will be the new configuration options you’ve got (default in brackets):

In view.get.config.xml:

  • truncateLength (100): Number of characters used to truncate event description in the Agenda view
  • weekView (“agendaWeek”): Used to specify which FullCalendar view to use for Week View, currently only “agendaWeek” is supported.
  • dayView (“agendaDay”): The name of the FullCalendar view to use for the day view. Currently only “agendaDay” is supported.
  • monthView (“month”):  The name of the FullCalendar view to use for the month view. Currently only “month” is supported.
  • weekMode (“variable”): Controls how the weeks will be rendered in a month view, maps to FullCalendar’s weekView option.
  • weekends (true): This shows Saturday and Sunday in Day, Week and Month views. Set to false to hide these days.
  • allDaySlot (true): Show the all day area at the top of day and week views. If set to false, then All day events will not be shown in those views.
  • firstDay (1): Which day does the week start on? 0 = Sunday, 1 = Monday etc. Affects Day, Week and Month view. (see also: calendar.widget_config.start_weekday i18n message string in slingshot.properties for setting the YUI mini/popup calendar)
  • minTimeWorkHours (7): The first hour displayed by default when in Day or Week views
  • maxTimeWorkHours (19): The last hour displayed by default when in Day and Week views
  • minTimeToggle (0): The first hour displayed when Working Hours display is off in Day and Week views.
  • maxTimeToggle (24): The last hour displayed when Working Hours display is off in Day and Week views.
  • aspectRatio (1.5): Controls the width:height ratio of the grid in Month views.
  • slotMinutes (30): Number of minutes between each line on the Day and Week views.
  • disableDragging (false): Should modifying events by dragging them be prevented?
  • disableResizing (false): Should modifying events by resizing them be prevented?

In create-event.get.config.xml

  • enableDocFolder (false): Should the “DocFolder” browser be included on the event creation page? This is mainly used for integration with Microsoft’s Meeting Workspaces.
  • defaultStart (12:00): Default time for a timed event to start.
  • defaultEnd (13:00): Default time for a timed event to end.

Future plans:

Now that this upgrade has been made, I’m hoping that in future versions we’ll be able to add new features and functionality to make the calendar even more useable. Ideas we currently have are: proper time zone support (currently events are stored with a fixed GMT offset, which mostly works except around DST), recurring event support (events created externally should be displayed correctly, but we don’t currently support creating recurring events), a user calendar (combining events from all sites and other sources) and the ability to input event times without having to use a 24hour clock.

Will the new calendar make a difference to your use of share? Is there anything else you would like to see included in future development work? Let me know in the comments.


Alfresco Home | Legal | Privacy | Accessibility | Site Map | RSS  RSS

© 2012 Alfresco Software, Inc. All Rights Reserved.