KNotations

After many, many delays due to illness and distraction, we're going to start putting together a practical roadmap for KNotes releases. In this post I'm just annotating an earlier post I made in February to point out what got done and whether priorities have changed. Bizarrely I note that the date posted is just a few days before I fell ill, but that the first tasks in the list did get done. I must have been pretty effective in those days :o).

Here is a hastily thrown-together list of issues I feel ought to be resolved before we publicise knotes. I'll be adding to this list, and annotating it, over the next few days.

OK - the list I made then was:

• Cleaner css/image skinning for weblogs - DONE - but user requests and practice since then have forced us to add 'skinning interface' to the release-1 features
• Complete fast_folders' move of 'no-wrap' from custom main_template to custom skin_path - STILL ESSENTIAL, JUST NEEDS SOME CAREFUL APPLICATION OF AN EXISTING SOLUTION
• Complete the migration to fast_folders as view for KNDiscussion - AFTER VACILLATING ON THIS I NOW AGREE THAT IT IS ESSENTIAL
• Complete the discussion-view for weblogs - DITTO

So, the top item on February's list got done (in a day or so :O), but we've since had so many demands for a skinning interface (other than the ZMI + ZPT+CSS knowledge) that I feel I must add some basic support for end-user skinning before we call KNotes 'released'. I'll write some notes on what 'basic' skinning support means in another post - choosing a stylesheet from options, setting (ugh!) background colours and banner images TTP, etc

The other items remain important, and remain undone. The fact that they remain undone is not surprising. I've only been back on my feet at all for a month and a half, and that has been totally occupied with catching up with project work, essential writing and putting out admin fires. Oh - and getting up to speed on e-portfolios and social bookmarking / social software integration issues. Weirdly, in the three months I was ill, the state of internet applications for the masses seems to have ratcheted forward a good big step after years of what looked like frustrating inertia to me :o)

One final point on those pre-release to-dos from February. At the time, I had planned to create a fallback, plain-html, accessible discussion view for the blogs and discussions, but to throw most effort into a fast javascript-savvy client loading data only and allowing many contextual actions without overall pageloads. In the months since, I had started to worry that the work to complete the javascript-savvy interfaces was - though 98% done - still a post-release issue. Since then, though, our users have in effect demanded faster and more convenient interfaces. Also, the 'AJAX' webapp style has begun to get some hype and support, which has made me less apprehensive about the accessibility issue. We will have a totally javascript-immune, accessible to all devices interface for browsing discussions and replying to them. But for early releases I am willing to keep that extremely crude and to put all the effort into a really fast and transparent application-like discussion interface.

More to come. Steve is back from Canada this week; we'll try to make a do-able and understandable roadmap within a few days.

Yesterday I posted about planning a 'release version' of knotes. I'm going to try to post at least once every day on this subject - heaven knows there is enough to write about! We use the FirstClass groupware system for our intranet. I send hundreds of messages a week to firstclass 'conferences' we've set up for documentation. I'm going to try to post some of those here instead.

As an aside, though, let me re-state the bleeding obvious: if you want to install KNotes in its current version, please do. We employ KNotes in as lot of sites now, and could not imagine living without it. But if you can wait a while, you might find life a bit easier. In either case, if you are thinking of trying KNotes at some point, please let us know.

We have to make some decisions about what exactly makes an issue a pre-release one. We already know of a lot of issues, minor bugs and feature requests, and we already have extensive plans for improving and extending KNotes. Most of those issues, bugs, features and improvements should be made post-release, but some of them would be much better completed before too many other sites were deploying KNotes in anger. I think these can be roughly categorised as follows:

Steve is just back from a refreshing holiday touring the Canadian Rockies in a big campervan with Rachel, Jade and Elle. I missed him. Already in the past day and a bit he's eliminated some nasty API bugs that had snuck in during a feature-push just before he left.

I think we're spotting bugs pretty quickly now, thanks to the volume of activity in our own project blogs and discussions. And when it comes to the subtleties of the XML-RPC API we're getting much better at predicting and avoiding issues. The date of a safe release is getting closer and closer!

The version of KNotes on sourceforge had fallen behind the version we deploy in our own production portals. Steve has updated it and fixed the problem that had prevented it being updated automatically. Somehow we'd removed CVS information from the product filespace during a big synch-with-experimental version changes in May.

Sorry to say we've been neglecting the sourceforge area; we'll be paying more attention again now. For instance, Steve just discovered that there were more than 190 downloads during the period of my illness. Scary! If anyone reading this has downloaded the tarball and not been able to get anything sensible working, please contact me (contact link in the navigation bar at top of page).

Steve and I had a very good meeting this morning to plan a 'roadmap to the roadmap'. The purpose was to make a plan for what to do in the vert short term in order to be able to publish a roadmap to release KNotes. Our hope is to be able to release a very stable, fast and feature-complete version within a month or two, but we need to do the analysis and scheduling very closely to be sure.

We reviewed a lot of issues. Most of them were quite small and straightforward to schedule ( if perhaps time-consuming in their plenitude :O) Some issues we identified as being big enough to require careful analysis before we can know how long they will take or whether they are realistic pre-release goals. These we decided to work hard on over the next week, and then review the scheduling implications when we are freshly familiar with the details.

• Javascript / PSWin for discussing and editing: repair, enhance or ditch
• basic top-level discussion views: need better plain-html views for discussion
• javascript enhanced discussion views: but we are close to having very convenient discussion interactivities
• user-profiles: we need sweet interfaces to give a sense of community, presence and portfolio to KNotes (and thus to Plone :O)

The first three items we need refreshing on because no-one has touched them since January. The fourth - user profiles - we have not yet done any coding for, but have been doing much background research in the past few weeks. Watch this space for more news as we get to know these issues in detail (again).

We've been hammering at the bigger pre-release issues for the past few days in order to scope in detail the work they entail. There is one 'new' issue among the big pre-release to-dos: user Profiles. We've made some progress on creating and maintaining your profile, and on integrating a view of one's KNotes and portal-wise activity with summaries and views of one's other activities using social-software services like del.icio.us and Flickr.

We're hoping to show off some mockups next week, but may have to put that off for a week or two since we're trying to concentrate on work-scoping spread over all the other big issues as well.

Our profile work has three motivations:

• KNotes itself clearly needs such a feature (though we had not previosuly thought it needed the feature immediately)
• Plone portals are in desperate need of better tools for giving members a sense of presence and community
• KnowNet has become involved in some exciting research in which KNotes profiling and presence features could play an important role. We are very interested in the idea of personal learning environments, and on personal development planning/profiling tools, and the way both of these can interact at different levels of globality with the many emerging social-software services (this kind of proposed tool is usually called 'e-Portfolio', but there are other meanings of 'e-Portfolio' to beware of).

There are three areas of work and research we'll be undertaking: First, get a satisfying sense of people-presence into KNotes, and use that to expose a sense of community in Plone portals; a large part of that work is about providing an interface for viewing member activity and encouraging interactions among the members (for instance proposing topics of interest for birds of a feather). Second, open the profile to interact with some of the well known social-software services that users might also make use of: shared bookmarks, references and photos for instance. We're looking into using the del.icio.us API, for instance, to bring a user's top N tags into their KNotes profile display so that viewers of the profile can at-a-glance see what kinds of thing that member is bookmarking.

The third area of upcoming work is subtler, but potentially very exciting. It is not a pre-release issue, but I want to briefly document it here since it will be colouring our approach to profiles: Though the social software services are getting massive user-ship (and press), they are still unknown to the kinds of end-user we usually work with. I suspect that this will be true for some time across most community portal constituencies. We want to explore having KNotes provide a kind or proxy or gateway into the wider world of 'well-known-to-techies' services.

This could be very useful not only for soliciting activity from ordinary end-users and spreading the meme of social-software, but also for exploring ways in which communities of interest can keep some special-interest context in their 'social' software environments. The global services cannot provide rich or contextualised metadata, for instance. The example I like to give is very simple: the National Guidance Research Forum (NGRF) is one of our flagship portals. We're in the process of converting 1400-odd academic references from our own Annotated Reference format into 'real' bibliographic shared references via connotea (NGRF account) and citeUlike (NGRF account). These services - like all the other global ones - have very simple, unconstrained and uncontextualised 'tagging' systems. Many of the references in the NGRF content are about 'counselling' - which in that context means careers guidance counselling. But if we simply tag these references "counselling" they will be conflated with the more common meaning/context of psychological counselling. There are many such issues which I suspect will start to demand attention as more people begin to use tools like connotea and del.icio.us "in anger"; we're very interested to see how an environment like KNotes can provide access to global services but preserve some special contexts and tools while doing so.

Another aspect we'll be very excited to explore is using KNotes to help users to build up their own 'e-Portfolios' for non-formal learning, personalised learning, personal-development planning etc. And, finally, there is the exciting issue of Profiles for Communities/Portals (as opposed to individuals)!

That third phase will - of course - be taking place post-release!

Steve has spent a lot of time and effort implementing the blogging APIs for KNotes, and done a great job of it: us authors can happily move between API clients like ecto and through-the-web (TTW) editing in a seamless way. But there are some aspects of our functionality where the API is just fundamentally so different from the web interfaces that issues spring up. One of those is the following:

• If I attached a file through the API, it ended up in the blog_mm foldr for the weblog the entry is in.
• If I attach a file through the web interface, it is created directly within the weblog entry as a Zope container object.

Why? Because of the way the API sends the files up to the server. From the client's point of view, it needs to send the files up before it can post the entry in which the files appear: until the server gets back to it, the client cannot know what the URLs for the attached files will be. So when an API client posts and entry which has a file attachment, it first posts the attachments. But this means that when our server-side of the APi gets the request to create the file attachments on the server, it does not yet have a weblog entry to put them into! Thus, we decided to follow blog-publishing practice and stash these in a folder called 'blog_mm' inside the weblog object.

But in our through-the-web interface, it makes much more sense to create these files directly inside the blog-entry object. Again this is a fundamental and natural consequence of the way Zope works and the way weblogs and web interfaces work.

OK. We have file attachments in two different possible locations. We also have information about them in an SQL table to enable very fast rendering. But that meands that every time we write a render we hav two cases to take care of. Ugh! For instance, requests have been coming in for some of that podcasting stuff everyone's been haring about. Easy-peasy - except for the two-faces-of-attachments :o)

So - yesterday Steve implemented a system on our test server which moves files from blog_mm into the weblog object when an entry is posted via the API. It was tricky ( don't ask :o) but it works. So we can now attack all the render efficiency issues in a clear an simple way. One such issue i to deliver file enclosures in feeds - RSS-2 and/or atom. That will be quite easy now; the only issue is that RSS-2 apparently only allows one enclosure, which mean that the natural approach ( any file attachment becomes an emclosure) will not work. We'll post sometime soon with news about which way we fell on that issue, and about how to podcast with KNotes :o)