KNotations

The only definition I ever found that created the lighbulb moment I was feeling was “Social software is stuff that gets spammed.” Not a perfect definition, but servicable in its way.

I hate spammers and dirty, venal link-farming vandals. I really, really hate them.

But they are out there, and they have noticed how useful trackback 2-way linking has been in raising the google page ranking of well-connected weblogs.

The link-farming spammers may also have noticed that the boost to the page ranks of well connected weblogs has, through the development of blogging, been a good thing for seekers after good content and for google: it helped people to find what they were looking for by googling it, and it helped them to browse around communities of discourse in fascinating and useful ways. They may have noticed this, but it has not stopped them from adding noise, aggravation and ugliness to the system by perpetrating trackback-spam.

In this post, I briefly explain trackback spam, and outline the measures we are planning to help the managers of KNotes content to resist the spammers by keeping spam links out of their own content. Click the permalink ("Continue reading this item") for more...

One of the main aims of knotes is to create good interfaces for resources: for the collection, annotation, discussion and tagging of links, files etc which members want to share among themselves or provide as a repository service. So far, we've had to concentrate hard on the basics of blogging in collaborative contexts, but for the big port of the SIGOSSEE site, we realised that we had to provide at least the beginnings of a resource-centric interface.

See the Open-Source in Education - Links and Resources weblog for an example. There is much work to be done yet, of course. We have not done anything about file resources, or for special search interfaces for 'resources'. But this is a step forward which we're hoping will allow SIGOSSEE members to really start adding and sharing link resources.

Those who follow this blog know that we've been planning to expand the range and features of the syndication feeds from Knotes content. The biggest priorities were:

filecasting ('podcasting'... doc-casting)

- delivering file attachments as enclosures ready to download directly from a news reader. In the case of audio or video files, this is usually called 'podcasting'

'full' content feed

Many users now prefer to experience web content almost exclusively through their newsreader of choice; not only reading content without viewing it in a browser, but also invoking editors to bookmark or blog about that content without viewing it on the web. To accomodate this growing preference, many blog publishers now offer 'full content' feeds. In the case of KNotes content, which has a lead-in/summary, main body text and extended-text, we would assume that the extended text should be excluded from 'full' content in feeds, just as it is in our aggregate web views - we'll call this 'full-minus-extended-text' the "main content" of an entry.

RSS 2 support

KNotes has supported RSS 1 and atom formats from its content. We have never wanted to appear to take sides in the atom-vs-RSS contention, but supporting only RSS 1 and atom coiukd appear to be implicit support for atom as the format for future features, since RSS 1 is ad old format. RSS 2 is the RSS format being developed in the near future and includes many useful features which our feeds have not yet supported - we wanted to include support for RSS 2 from KNotes, and begin to explore support for the new features in it.

On the other hand, we want to avoid a confusing proliferation of choice about which feed a user might subscribe to. We already have a choice whether to 'include_discussion', anfd a choice between atom and RSS-1 - making 4 choices in all - if we delivered the complete range of RSS-1,RSS-2,atom X include_discussion X full-content,main-content,summary we would be giving users a choice of 18 feeds :o{

So we've decided that the RSS 2 feed will be 'the' main-content feed and 'the' filecasting feed. In future we might add these features to the atom feed (which is now beginning to feel featureless :o)

As yet, the RSS_2.xml feed does not support the include_discussion search argument, so is only available for level-one content. We'll add support for include_discussion later today, an then put links into the Subscribe sidebar to expose the new RSS 2 filecasting main-content feed to end users. It displays the lead-in anf the html main body, with enclosures for any file attachments in the entry. If there is extended text, this is clearly flagged at the top an bottom of the feed item (for instance "View full content (18405 bytes more)". See the screenshot.

In the meantime, you can get a feel for the new format either by appending '/RSS_2.xml' to a KNotes weblog url, or by trying one of these:

• SIGOSSEE Project News [main content + filecasts]
• KNotations [main content + filecasts]

Some policy decisions remain to be made - for instance, which of the formats should be the 'one' auto-discoverable feed which we link to from the header? I am inclined to think that the new RSS2 feed ought to be the 'one' that users subscribe to when they click the RSS button in safari, etc. We'll get back to that question later :o)

We've been asked several times about podcasting support out of knotes content, but so far have not had an opportunity to do anything about it. Frankly, until recently I was unconvinced about the power of podcasting. But for a while now I have been trying to get the point of it, and I guess I have (see for instance the recent news that the BBC is offering experimental podcasting feeds for some of its radio programming. It has also helped that Al Harris is working on a series of brief interviews on small enterprises and educational technology / open source software; these would make great audio feeds.

It has also occured to me that many of our users post a lot of documents (especially Mico=rosoft Word files :o) - podcasting in the sense of feeds with file enclosure information could be a great way to announce new documents and share them with subscribers.

OK. Sold. What kind of support should we offer? For a start, I do not think we need to proliferate yet another feed option in links from knotes pages. i think that enclosures should 'just work' by default in our feeds, if possible. Second, I do not think that editors should have to specify that an uploaded file should be made available through a feed - all files uploaded as attachment to weblog entries should by default be made available in the podcast feed.

We will be introducing RSS-2 feeds from knotes to support podcasting. We never intended to neglect RSS extension work in favour of atom, so RSS 2 support has been in our plans for ages. We may replace the existing RSS 1 feeds with RSS 2. We need to look hard at backwards-compatibility first; otherwise we will offer 3 feed types as salient links in weblog views: RSS 1, RSS 2 and atom.

We then have a thorny little issue to solve before it is straightforward for all attachments to be detected and added as enclosures. We want to solve this issue anyway for other reasons: at the moment attachments that come through API clients are stored in a weblog's/blog_mm folder, while attachments added through the web are stored directly within the weblog entry. The latter is the best way to treat attachments from the point of view of summarising and aggregating them as resources, and also for managin large numbers of them. the reason we stash attachments added through the API into blog_mm instead is that when the API gets the request to create an attachment the blog entry does not yet exist. We've considered a few hacks around this in the past but never had an urgent reason to do something about it. Now we do.

Watch this blog for news about the solution we come up with, and also for news about the RSS 2 feeds and podcasting.

We've been asked several times about podcasting support out of knotes content, but so far have not had an opportunity to do anything about it. Frankly, until recently I was unconvinced about the power of podcasting. But for a while now I have been trying to get the point of it, and I guess I have (see for instance the recent news that the BBC is offering experimental podcasting feeds for some of its radio programming. It has also helped that Al Harris is working on a series of brief interviews on small enterprises and educational technology / open source software; these would make great audio feeds.

It has also occured to me that many of our users post a lot of documents (especially Mico=rosoft Word files :o) - podcasting in the sense of feeds with file enclosure information could be a great way to announce new documents and share them with subscribers.

OK. Sold. What kind of support should we offer? For a start, I do not think we need to proliferate yet another feed option in links from knotes pages. i think that enclosures should 'just work' by default in our feeds, if possible. Second, I do not think that editors should have to specify that an uploaded file should be made available through a feed - all files uploaded as attachment to weblog entries should by default be made available in the podcast feed.

We will be introducing RSS-2 feeds from knotes to support podcasting. We never intended to neglect RSS extension work in favour of atom, so RSS 2 support has been in our plans for ages. We may replace the existing RSS 1 feeds with RSS 2. We need to look hard at backwards-compatibility first; otherwise we will offer 3 feed types as salient links in weblog views: RSS 1, RSS 2 and atom.

We then have a thorny little issue to solve before it is straightforward for all attachments to be detected and added as enclosures. We want to solve this issue anyway for other reasons: at the moment attachments that come through API clients are stored in a weblog's/blog_mm folder, while attachments added through the web are stored directly within the weblog entry. The latter is the best way to treat attachments from the point of view of summarising and aggregating them as resources, and also for managin large numbers of them. the reason we stash attachments added through the API into blog_mm instead is that when the API gets the request to create an attachment the blog entry does not yet exist. We've considered a few hacks around this in the past but never had an urgent reason to do something about it. Now we do.

Watch this blog for news about the solution we come up with, and also for news about the RSS 2 feeds and podcasting.

This is just a test of del.icio.us playtagger - the new shared javascript from del.icio.us which enhances mp3 links with a lightweight flash object to play the audio, streamed, in the browser, in your page. The enhancement also places a tag-me link to post the mp3 to delicious.

This could be a great way for ordinary endusers to get the best of both podcasting and in-page playing: posting mp3 files is the open, standards-based way to share audio, allowing real RSS podcasting, permalink-sharing, user-timed listening etc. But some authors like the in-page feature that an embedded flash object provides, and seem unaware of the advantages of simply posting mp3's from the user's point of view. For an example of audio which will play in the page via Flash, but otherwise cannot be used, see this page of 'podcasted' interviews

If this little delicious shared javascript trickery works, it should mean that just attaching an audio file to a knotes blog post makes it playable in the page and usable as a file if the user wants to save it, link to it, or subscribe to podcasts.

I'm attaching a little audio file: please forgive the copyright-busting; if you enjoy it, do by all means look Catfish Keith up and buy some CDs :o)

If this works, you should be able to do streamed listening to the audio attachment (see above for the link - it should have a little arrow nexk to it, like a forward button). I may have to change some templates to get the script include into the head; for this example I'm first trying the script include in the body of this message. I looked over the 3289 bytes of the include and it does not look to me as if it needs to be in the page header.

This is just a test of del.icio.us playtagger - the new shared javascript from del.icio.us which enhances mp3 links with a lightweight flash object to play the audio, streamed, in the browser, in your page. The enhancement also places a tag-me link to post the mp3 to delicious.

This could be a great way for ordinary endusers to get the best of both podcasting and in-page playing: posting mp3 files is the open, standards-based way to share audio, allowing real RSS podcasting, permalink-sharing, user-timed listening etc. But some authors like the in-page feature that an embedded flash object provides, and seem unaware of the advantages of simply posting mp3's from the user's point of view. For an example of audio which will play in the page via Flash, but otherwise cannot be used, see this page of 'podcasted' interviews

If this little delicious shared javascript trickery works, it should mean that just attaching an audio file to a knotes blog post makes it playable in the page and usable as a file if the user wants to save it, link to it, or subscribe to podcasts.

I'm attaching a little audio file: please forgive the copyright-busting; if you enjoy it, do by all means look Catfish Keith up and buy some CDs :o)

If this works, you should be able to do streamed listening to the audio attachment (see above for the link - it should have a little arrow nexk to it, like a forward button). I may have to change some templates to get the script include into the head; for this example I'm first trying the script include in the body of this message. I looked over the 3289 bytes of the include and it does not look to me as if it needs to be in the page header.

The release of a fully productised and robust core knotes has fallen a bit behind schedule recently because we've been very busy making use of it in some of the community portals we maintain, and in adapting it to new purposes. One very interesting use we've been trying out is as a framework for portal comments. Our users have always hated the default Plone talkback mechanism: 'Discussion Item' entries do not allow rich text, active links, attachments, or post-submission editing, and they tend very much to get orphaned. On the other hand, the idea of posting a comment is very easy and intuitive for most users, there is an affordance to do so right in the pages, and it provides a simple means for user to post their own writing in a prominent place (without understanding workflow, etc :o). We briefly tried using CMFBoard as a portal_discussion handler last year, but its treatment of the issue was profoundly misconceived, not to mention buggy and veeeery slow, and was disastrous in real sites. So we've made do so far with either turning off discussion_allowed, or with the robust but feature-poor default portal_discussion. Last week, we started experimenting with replacing add-a-comment with shared weblogging, and demonstrated this to some of the key editorial people in our main sites. For the first demos, all we were doing was displaying trackback details for pages instead of comments; as luck would have it, an interesting weblog post had been made about an article in the NGRF site, and the editors were blown away when they saw external commentary summarised in their page.

So we decided to go ahead with a very quick and dirty takeover of portal_discussion. We've done this in two new sites, and one site that already had extensive comments (we wrote a little script to make new weblog posts and replies to them from existing comments).

How to set up knotes for portal comments ( the quick & dirty way )

1 - make a new weblog to hold the comments

- This one is easy. Just add-item:WebLog wherever you can make Plone content. Of course, you'll probably want this somewhere prominent in the site content hierarchy, and prominently linked-to, perhaps from site_actions.

2 - give all site members contributor privileges to that weblog

- This is a bit tricky at the moment. It is very easy to give contributor permissions to a group or a set of members using the local role form (Weblog Contributor role), but giving all members the required permissions is a bit harder. We have implemented something similar for the moderation interface for discussions in knotes, but not yet for blog entries. You have to go into the ZMI for the weblog you want to allow all members to post to (not for the entire portal!), open the Security tab, and manually tick the 'member' column checkbox for the following permissions:

• Add Portal Content
• Add KNBlog
• Add KNLink
• Add KNExFile

3 - turn off the display of the 'Reply' content-action

We always hated this pesky content_action anyway, since it forces a box around content for logged in members who can only choose between the view and reply tabs. If you are going to replace talkback with blog-this-page by changing the viewThreadsAtBottom.pt template, you have to get rid of the competing action from default talkback machinery. In the ZMI for a portal, just open portal_discussion, then actions, and untick the 'visible' checkbox for the reply action. See the screenshot:

4 - customise the viewThreadsAtBottom template.

The viewThreadsAtBottom.pt template is what is called in page rendering by Plone to display the comments and add-comment etc content. A very crude, but working, replacement template is attached to this post: that template displays trackbacks instead of comments, and has as its form action for Add-a-comment the creation and editing of a new blog_entry, with initial link details filled in with the details of the page it is called from. A list of weblogs in the site which the member can contribute to is also provided; the comment will be posted into the chosen blog.

5 - import any existing comments

We have written a little python script to convert comments into weblog entries, and a little page template to call it from. The script takes as an argument the weblog to post into, which you enter from the form field in the rendered template. Run the convert_comments template FROM THE DOMAIN NAME OF YOUR SITE (eg www.my.domain.edu/convert_comments), and it will create weblog entries for all top-level comments (tracking back to the pages the comments were about), and making replies (as Discussion, not blog entries) to those new entries from any comments-on-comments threads. It will add entries in the members SQL table where needed. We've tried this in a production site, and it certainly seemed to work. It may take some time to run if you have many hundreds of existing comments - try it in a copy first!

6 - remove any aggregations you may have made of comments, for instance portlets

You probably do not have to worry about this, but if you have been displaying information about recent comments, or adding comments-on-items to listings, you'll have to take those out or change them to get their information from the trackback machinery instead of the talkback machinery (ie grabbing trackback entries from knotes' SQL methods instead of comments from portal catalogue :o) It is very easy, using CMFSin, to include a recent_comments portlet from the RSS for the weblog you use for comments. It is your choice whether to use the RSS with-discussion or without (knotes provides both options for syndication of blogs). Use the RSS rather than the atom feeds for CMFSin.

Ta-Da! In these simple steps, you've got a whizzy weblog behind your portal comments machinery. This makes it a lot easier to see comments as a whole, to post items which are not specific to a particular page, and to give a sense of community and activity. As a very useful side-effect, your site's pages will display any trackbacks they may receive from other sites, which can add a real sense of being part of a wider community of communities.

You can see this in action at the National Guidance Research Forum site. NB - this links to the NGRF Community Comments weblog instead of the NGRF top of site; I did not want to pollute the top-of-site trackbacks with links to technical posts like this (and knotes does not display top-of-weblog trackbacks yet :o)... please do not send trackback pings from technical blog entries to the NGRF top of site.

Within a couple hours, we'll have our production version of knotes in shape and suitable for deployment on other peoples' zope/plone installations. Huzzah!, phew!, and none-too-soon! :o)

We now have to carefully merge our production version with the CVS at the knotes sourceforge area, and make a new tarball to put in our own downloads areas, based on that CVS version. We expect it to take the rest of this week, given other demands on our time - though we can hope for speedier progress.

Once we have a stable and highly-usable release on sourecforge, we'll concentrate on improving documentation, and on providing issue-tracking etc - we'll make an open blog-forum for developers to post queries and share ideas as well

Of course we'll also have to do some writing at our own product pages :o) I'd also very much like to get some time to do justice to the knotes.net site, which is an utter mess now. I want to use knotes.net as a text-case for a plone sitelet theme / customisation-policy, comprised transparently and attractively of just a blog or two and a few simple downloads/content areas (inspired by particletree.com).

I'll post a note here when we've upgraded the CVS version of knotes.

Last week, we completed pushing the recent improvements to knotes into the CVS version and tarballs at the knotes sourceforge area. We've been intensively testing in our own user communities as well as in a variety of test installations. It seems to just work, and end-users seem to just get how to use it, so we feel we're approaching a really useful product. To reflect this, we've incremented the version number to 0.8 BETA. Well done and many thanks to Steve and to our beta testers!

Still to-do are the writing of better admin documentation and end-user help, and the provision of developer forums, issue-tracking, etc. We're attending to those requirements as quickly as other work allows.

We are confident now that other Plone admin folks should be able to install knotes easily and make good use of it to add dynamism, discussion and user-expression to Plone sites. Please do try it out and let us know of any issues you encounter.

We're also confident that knotes with Plone provides a rich and flexible platform for experimental projects looking into variations on the theme of blogging - into uses in learning and community building where the basic behaviours and building blocks of blogging need to be enhanced or contextusalised. That's why we built it, and we're already starting to make progress with some ideas of out own that make use of knotes 'plus'.

If you are an educational researcher or community developer, please get in touch with us and share your ideas; we may be able to help. We're particularly keen to explore lightweight services-oriented approaches in the "web2.0" style - adding microformats, integrating with social bookmarking, etc. We are also eager to enhance the profile-building and user-glu aspects of the knotes member-profile, and to explore the use of knotes + Plone for e-portfolios and personal learning environments.

Have a look at this new testbed in the NGRF site. It's where ourselves and the editorial team from the NGRF are going to be hammering hard at the discussion system issues over the next few days.

Welcome beta testers to a set of new weblogs for the NGRF. During late November 2005 we're using these weblogs to work out a new interface for discussion, and new looks for the weblogs. Please bear with us if you encounter problems using them.

Already there are some important improvements:

member profiles

most of the features needed for basic profiles are working. Names of authors/commenters are links to their profiles in weblog views. This demo includes the display of some more advanced features using static data for display. If you fancy having a go yourself, the template names work in all the sites including the knownet site: $portal_url/Members/my-username/profile to view, $portal_url/profile_edit_form to edit your own (yes, the edit-form needs a bit of guide text and formtting - we'll be adding / changing fields so hav not settled that form yet).

reply from within blog one-entry view

there is a simple comment form directly within the one-entry blog view, with AJAXian log-in formlet for convenience.

better context in the main RSS 2 feed

discussion items in the RSS 2 feed now have a tagline pointing to the item they reply to. The node in the RSS 2 feefd now points to a usable interface (for blog entries only as of today, discussion items to follow shortly)

pretty-printing of level- comments on blog entries

this has been the case in my own elearning2.0 blog for a couple weeks... much nicer display of comments and trackbacks on blog entries in blog one-entry view

access to experimental 'blog-forum' ajaxian discussion forum view

the main banvigation links now include 'forum' which points to a new forum-view for the weblog. I'll post a lot more detail about it when it's a bit more mature, but do try clicking on the icons at the far left of rows to fetch the row's subcontents/replies, an do explore the ajaxian batching and blog-like navigation ... actually, ince there is so little demo content in those weblogs, try this link instead for blog-forums - experimental - for KNotations weblog.

Over the course of today we'll be concentrating on the blog-forum views so that we have a good permalink for discussion items, and also try to 'enliven' some of the member profile features which sre represented by static data in the NGRF demos.

We've been remiss in keeping the documentation of progress flowing into this weblog. Apologies for that - but it's durned hard to do it and write about it at the same time :o) - So I'm going to try re-using content to help spread the word. I posted into the site-editors weblog for the National Guidance Research Forum this morning with a list of pending and in-progress to-dos. Most of these will also become core KNotes improvements, so are worth noting here.

This post is a copy of an email I sent this morning, briefly documenting some small and large jobs we agreed in the meeting in Bangor but which have not yet gotten attended to or are still in progress

My brilliant colleagues at ONE/Northwest and our talented collaborators at LightSky Designs and RagingWeb just helped Snow Leopard Trust launch their new website.

It's a beautiful site, with a ton of great content and some eye-popping photos of these magnificent cats and the communities they live among...

...Like all of the websites we do, Snowleopard.org is powered by Plone, the most powerful and easy-to-use open-source content management system around. Plone makes it easy for Snow Leopard Trust staff to maintain a large, complex site.

It is not often that I encounter a Plone site that really delights me with great design. In fact, once you've been developing in Plone for a while, most Plone sites look pretty same-ish and un-designed. The work, talent, and judgement that has gone into snowleopard.org is exemplary: great information design, great graphical and interface design, and some well-judged software and integration.

An example to us all :o)

Well done, everyone involved!

Thanks to generous support from the Leona M. Geyer Charitable Trust, we are proud to present a completely new www.snowleopard.org! Here at the Snow Leopard Trust we've been burning the midnight oil to bring you this new website, jam-packed with information about our conservation programs, the scientific research we support and carry out, and all our other activities.

But wait, there's more! If you've always wanted to learn more about the countries where we work, the people who live there, or snow leopards themselves, then this is the place for you! Stick around for a few minutes and explore...

By the way, I've begun to collect bookmarks to interesting or unusual plone sites in my del.icio.us "plone-site" tag.

My brilliant colleagues at ONE/Northwest and our talented collaborators at LightSky Designs and RagingWeb just helped Snow Leopard Trust launch their new website.

It's a beautiful site, with a ton of great content and some eye-popping photos of these magnificent cats and the communities they live among...

...Like all of the websites we do, Snowleopard.org is powered by Plone, the most powerful and easy-to-use open-source content management system around. Plone makes it easy for Snow Leopard Trust staff to maintain a large, complex site.

It is not often that I encounter a Plone site that really delights me with great design. In fact, once you've been developing in Plone for a while, most Plone sites look pretty same-ish and un-designed. The work, talent, and judgement that has gone into snowleopard.org is exemplary: great information design, great graphical and interface design, and some well-judged software and integration.

An example to us all :o)

Well done, everyone involved!

Thanks to generous support from the Leona M. Geyer Charitable Trust, we are proud to present a completely new www.snowleopard.org! Here at the Snow Leopard Trust we've been burning the midnight oil to bring you this new website, jam-packed with information about our conservation programs, the scientific research we support and carry out, and all our other activities.

But wait, there's more! If you've always wanted to learn more about the countries where we work, the people who live there, or snow leopards themselves, then this is the place for you! Stick around for a few minutes and explore...

By the way, I've begun to collect bookmarks to interesting or unusual plone sites in my del.icio.us "plone-site" tag.

My brilliant colleagues at ONE/Northwest and our talented collaborators at LightSky Designs and RagingWeb just helped Snow Leopard Trust launch their new website.

It's a beautiful site, with a ton of great content and some eye-popping photos of these magnificent cats and the communities they live among...

...Like all of the websites we do, Snowleopard.org is powered by Plone, the most powerful and easy-to-use open-source content management system around. Plone makes it easy for Snow Leopard Trust staff to maintain a large, complex site.

It is not often that I encounter a Plone site that really delights me with great design. In fact, once you've been developing in Plone for a while, most Plone sites look pretty same-ish and un-designed. The work, talent, and judgement that has gone into snowleopard.org is exemplary: great information design, great graphical and interface design, and some well-judged software and integration.

An example to us all :o)

Well done, everyone involved!