KNotations

Which RDBMS? :: PostgreSQL, MYSQL

Knotes was built for speed, that's why it's been built using a RDBMS. During the development stages we used Postgresql throughout, this has led to some small non-standard, or shall I say 'non-MYSQL' code sneaking into some of the pre defined queries. This isn't a big issue and we already have a 'MYSQL friendly' version available. This just needs some thorough testing before being merged into the product.

Support for Plone 2.1

KNotes stores all related data (attachments,links...) in the RDBMS. To enable links and attachments to get their data storerd in the RDBMS, we created special versions of the standard 'Link' & PloneExFile products. The version of PloneExFile used for subclassing is now quite old and newer versions are not supported, this old version will *not* install in Plone 2.1. We will therfore drop PloneExFile as a dependency from any future release.

ZAttachmentAttribute is required by PloneExFile, even when the dependency on PloneExFile has been removed we still need ZAttachmentAttribute for our own KNExFile. The version of ZAttachmentAttribute used at the moment (2.3) does not install without a few minor hacks to the installer script. Altough these are *very* minor, they are still of some concern.

Apart from the problems outlined above, KNotes versions 0.7 alpha and above *will" install and work in Plone 2.1

I'm about to post a short overview of our plans for dealing with trackback spam, and realised that before going into those measures, I should first review the behaviour - current and planned - regarding authenication for RSS/atom in KNotes.

First - what is the issue? Basically, manegers can use the permissions form for a KNotes weblog to make the blog readable only to certain members. Likewise, a parent Plone folder could have been given a workflow state of 'private'. In either case, through-the-web pageloading requires authentication by a member with the correct permissions.

But what about the RSS/atom feeds for that private, members-only weblog? If a snoopy and savvy person wanted to type in the url for an RSS feed for the private weblog, surely they should not be able to read content in their news-reader which they could not read in their browser?

No, they should not. And at least some news-reading clients respect that. For instance, NetNewsWire offers username/password properties for a subscription, and will present an interface for entering them if the subscription demands it with its http response header.

And KNotes' RSS and atom feeds will not return content unless given an appropriate username/password when the request is made on a zope object which would require authentication for viewing through the web. Try it... if you subscribe in netnewswire to a private weblog, you'll have to enter a username an password ( in the get-info dialog for the subscription ) in order to fetch content.

BUT there is still work to be done:

Demand authentication nicely

We need to have the private feeds send back an authentication demand header rather than the error they currently do. This is a very small job but needs to be scheduled,

Return '' for nested private content in public feeds

More important by far: If you subscribe to a KNotes feed with '?include_discussion=1' -- ie you want to get nested content in the feed -- you can read nested private content. At the moment, privacy is only 'ert' at the level of the object the feed is called on. The content of the feed is assembled with an SQL query, so zope-wise permissions are not taken into consideration when grabbing the item content (and we definitely want the SQL speed). What we need to do is to impose a very simple policy: content which is not public through the web should not appear as nested content in any feed.

This policy would be draconian but safe. The SQL database kndiscussion table rows can 'know' whether or not "some" authentication is required (but cannot of course encapsulate zope's complex acquirable permissions, so cannot know whether the current request should authenticate against a row). But, since a row can know that its content is not public, it can have its feed content an empty string except when called directly on its parent object... in other words, a different query would have to be called for the non-nesting case

As you can see, some changes to the SQL data model are required in order to prevent non-public content from appearing in feeds other than those called directly on its parent. That means work, and will have to wait.

In the meantime, beware that private content could be sniffed by savvy snoopers. Personally, I would resist privacy anyway, but I appreciate that it is very important to some of our own users - and we will attempt to effect correct behaviour soon after 'release' :o)

sigh...

Auto-detection of the edit API hads been a real pain for us. Clients differ in their behaviour.

And recently ecto, which used to auto-detect the preferred API and access-point for KNotes blogs, now fails. And blogjet, which used to fail, succeeds :O)

... we did not change anything I am aware of, so I suspect these are clientside changes :o{

ANYway, the worst part of not auto-detecting is that the user is not currently given a hint to specify the MT API (Movable Type). The MT API allows some clients - ed ecto - to edit the summary and extended text, pluys other goodies I think. Which makes for more useful content structuring from the editor. We're proud of the work we did implementing the serverside for the API for plone, and it did take work to support those neat MT extenseions - so this is frustrating :O)

So...

1. If you are setting up remote editing for a KNotes blog, and auto-detection of preferences and API access-point does not work, please choose the MOvable Typ APi if given the choice.
2. We should remember to have another look at the way we specify the MT as preferred API in our edituri text
3. It seems likely that whatever is preveting ecto from noticing our preferred API might also be the culprit in preventing it from auto-detecting the API attachment
4. Andin the meatnime we ought to add some text to the 'set up external editing' content to explain why the MT APi is better.

... of course, once KNotes becomes well-known by the developers of the clients, these problems start to become theirs as well as ours :O)

The only definition I ever found that created the lighbulb moment I was feeling was “Social software is stuff that gets spammed.” Not a perfect definition, but servicable in its way.

I hate spammers and dirty, venal link-farming vandals. I really, really hate them.

But they are out there, and they have noticed how useful trackback 2-way linking has been in raising the google page ranking of well-connected weblogs.

The link-farming spammers may also have noticed that the boost to the page ranks of well connected weblogs has, through the development of blogging, been a good thing for seekers after good content and for google: it helped people to find what they were looking for by googling it, and it helped them to browse around communities of discourse in fascinating and useful ways. They may have noticed this, but it has not stopped them from adding noise, aggravation and ugliness to the system by perpetrating trackback-spam.

In this post, I briefly explain trackback spam, and outline the measures we are planning to help the managers of KNotes content to resist the spammers by keeping spam links out of their own content. Click the permalink ("Continue reading this item") for more...

Those who follow this blog know that we've been planning to expand the range and features of the syndication feeds from Knotes content. The biggest priorities were:

filecasting ('podcasting'... doc-casting)

- delivering file attachments as enclosures ready to download directly from a news reader. In the case of audio or video files, this is usually called 'podcasting'

'full' content feed

Many users now prefer to experience web content almost exclusively through their newsreader of choice; not only reading content without viewing it in a browser, but also invoking editors to bookmark or blog about that content without viewing it on the web. To accomodate this growing preference, many blog publishers now offer 'full content' feeds. In the case of KNotes content, which has a lead-in/summary, main body text and extended-text, we would assume that the extended text should be excluded from 'full' content in feeds, just as it is in our aggregate web views - we'll call this 'full-minus-extended-text' the "main content" of an entry.

RSS 2 support

KNotes has supported RSS 1 and atom formats from its content. We have never wanted to appear to take sides in the atom-vs-RSS contention, but supporting only RSS 1 and atom coiukd appear to be implicit support for atom as the format for future features, since RSS 1 is ad old format. RSS 2 is the RSS format being developed in the near future and includes many useful features which our feeds have not yet supported - we wanted to include support for RSS 2 from KNotes, and begin to explore support for the new features in it.

On the other hand, we want to avoid a confusing proliferation of choice about which feed a user might subscribe to. We already have a choice whether to 'include_discussion', anfd a choice between atom and RSS-1 - making 4 choices in all - if we delivered the complete range of RSS-1,RSS-2,atom X include_discussion X full-content,main-content,summary we would be giving users a choice of 18 feeds :o{

So we've decided that the RSS 2 feed will be 'the' main-content feed and 'the' filecasting feed. In future we might add these features to the atom feed (which is now beginning to feel featureless :o)

As yet, the RSS_2.xml feed does not support the include_discussion search argument, so is only available for level-one content. We'll add support for include_discussion later today, an then put links into the Subscribe sidebar to expose the new RSS 2 filecasting main-content feed to end users. It displays the lead-in anf the html main body, with enclosures for any file attachments in the entry. If there is extended text, this is clearly flagged at the top an bottom of the feed item (for instance "View full content (18405 bytes more)". See the screenshot.

In the meantime, you can get a feel for the new format either by appending '/RSS_2.xml' to a KNotes weblog url, or by trying one of these:

• SIGOSSEE Project News [main content + filecasts]
• KNotations [main content + filecasts]

Some policy decisions remain to be made - for instance, which of the formats should be the 'one' auto-discoverable feed which we link to from the header? I am inclined to think that the new RSS2 feed ought to be the 'one' that users subscribe to when they click the RSS button in safari, etc. We'll get back to that question later :o)

We've made good progress on tools to make trackbacks easy to monitor and manage. By the end of the week, we'll have deployed a pretty thorough suite. In tests on our own content, these make it easy to bash trackback spams almost as soon as they come in.

There are two new through the web templates:

weblog.trackbacks

This is a weblog-specific template which can be invoked on a weblog or any content within it. It displays a batched aggregate weblog view of the trackbacks within that content, with affordances to delete trackbacks if you have manager role. This template will soon become the 'more' link from the 'recent trackbacks' sidebar, and we'll also hav a link to it from the stats sidebar

trackback_admin

This is a Plone template which can be invoked on a Plone portal itself, or on any content within it. It requires manager role. It provides a batched listing of all trackbacks within the content (either flat or deep) with checkboxes for selecting items to delete

There is also a new RSS feed - TBs.xml - which can be invoked on any Plone or KNotes content. It demands authentication by a member with manager role, anf takes an optional limit search argument. It lists (deep) trackbacks anywhere ithin the content it is invoked on. The item display includes links to the content rceiving the trackback, the URL source for the trackback, and - especially convenient - a direct DELETE THIS TRACKBACK link. Clicking the delete link in an item calls the delete script through the web ( and will demand authentication if you are not already logged in ). We have used this feed successfully in netnewswire on os-x but at the moment are not getting authentication to work in the windows readers we've tried. i we annot get this working in a widerrange of readers by the end of the week, we'll change our plan - if so, we'll post a notice here to that effect.

See the screenshot for an annotated anatomy of the special RSS feed.

Our colleagues in the NGRF gavea big hands-on workshop in Cyprus Friday August 26, and were concerned about performance. It can really exercise a server having a couple dozen logged-in users demanding frequent refereshes of dynamic content... as we learned to our embarassment in a workshop early in the spring :o)

Since the spring workshop we have made numerous techie performancee tweaks to the basic server technoology we run, but at the same time we had been amassing the features in KNotes.

KNotes began life as a very nippy application - in fact, performance was one of our number-one goals. But in the way of these things, as features grew so did render time. We were due a thorough review of the speed of rendering an the size of the markup deleivered. So we made that review in advance of the Cyprus workshop.

[ASIDE] - This is an extremely trivial note, of the sort that I write up as firstClass conference messages dozens of times every day. I'm posting it here because I'm trying to experiment with making our development notes more publicly visible, and also because in this case it kind-of documents the sidebar-collapsible feature and some design decisions which other developers ought to know about :O)

I find it annoying that the sidebars are all expanded while the page is loading, and then those which are cookie-d to collapse do so after page load. It makes an odd and suddent transition in what the user sees, and makes 'pre-scrolling' very confusing. In case you didn;t know, the sidebars are each collapsible (click the little box icon to the left to toggle this), and the preferences are kept as a cookie. See the screenshots for before and after views o the sidebars during pageload.

There is no need for this - we could easily set up actual display:none style rules before the page content loads instead of acting on the DOM nodes in javascript after pageload. That is, we dynamically write style rules into a little style block which come above the markup for the sidebars, and which declare that certain of the sidebars have the rule display:none applied to them ( the cookie specifies the id's for the selectors )

There are 2 choices for when/how to create the style rules:

1. analyse the cookie in zpt at the server, and write in the style rules in the markup, or
2. write them in javascript dynamically as the page loads in the client but before the sidebars are loaded

Option 2 is best since the cookie only makes sense if js is enabled; otheriwise css-on + js-off could hide content irrevocably

Also ZPT is darned awkward for writing style rules. It is easier to generate css rules as text in javascript and write them into the page than it is to use zpt :O)

I recommend having a bit of js-written styling just before the sidebars ( so the main content loads that little bit faster above them )

I'll do that tomorrow morning.

While I'm at it I'll have an initial look at having the categories sidebar written in js as a tree reflecting the categorical facets hierarchy. We found out today that very long category facet-chains cause a styling glitch in MSIE/win, and I've been meaning to make a js at clientside tree render / drop-down render for the categories for ages.

During PageLoad - all expanded	After PageLoad - some expanded

My brilliant colleagues at ONE/Northwest and our talented collaborators at LightSky Designs and RagingWeb just helped Snow Leopard Trust launch their new website.

It's a beautiful site, with a ton of great content and some eye-popping photos of these magnificent cats and the communities they live among...

...Like all of the websites we do, Snowleopard.org is powered by Plone, the most powerful and easy-to-use open-source content management system around. Plone makes it easy for Snow Leopard Trust staff to maintain a large, complex site.

It is not often that I encounter a Plone site that really delights me with great design. In fact, once you've been developing in Plone for a while, most Plone sites look pretty same-ish and un-designed. The work, talent, and judgement that has gone into snowleopard.org is exemplary: great information design, great graphical and interface design, and some well-judged software and integration.

An example to us all :o)

Well done, everyone involved!

Thanks to generous support from the Leona M. Geyer Charitable Trust, we are proud to present a completely new www.snowleopard.org! Here at the Snow Leopard Trust we've been burning the midnight oil to bring you this new website, jam-packed with information about our conservation programs, the scientific research we support and carry out, and all our other activities.

But wait, there's more! If you've always wanted to learn more about the countries where we work, the people who live there, or snow leopards themselves, then this is the place for you! Stick around for a few minutes and explore...

By the way, I've begun to collect bookmarks to interesting or unusual plone sites in my del.icio.us "plone-site" tag.

Funny how these little bells and whistles can seem o=unimportant until you start creating content yourself :o)

I've long known that we should do something about logging the pings hat the trackback machinery attempts to send out. A trackback 'ping' is a little request we make to a trackback-aware server's "trackback:ping" url for their content we are linked to. That request includes the data about our own content. The other server can, if so configured, use that data - for instance to display a link back to our content's commentary on theirs.

Oh my we've been busy - though sometime the hours of toil do not see to get enough done.

This is a note to document some issues dealt with so far today.

Do you know other sites that suport trackback ?? It's important to test the way other website could suport trackbacks ! I understand that this is an important feature, so maybe it's not so comumn to a website to have this killer function If anyone knows anything about it !

Testing trackback can be a pain if you do not already have access to another blog publishing system with trackback enabled. There is a publicly available test service, though - if only I could recall the URL. Steve has the URL, but it's the middle of the night and he's off ill anyway. I just googled for it and got too much noise in the results - aha! Just tried del.icio.us/tag/trackback and one such service was near the top. In the web2.0 world we have to start thinking different :o). I've collected that link into my own tag del.icio.us/Mike_Malloch/webtech/trackback as well, and will try to collect other useful resources there in the near future. The form is at No-Host Trackback.

I'll also trackback to this entry from some other sites (I am admin in a lot of them so can get away with that kind of abuse). And here are a couple examples of KNotes entries that have been linked to from outside KNotes ( scroll to the bottom of the entry to see trackbacks or click the 'just trackbacks' link. At some point I'll think to add an internal anchor in the main templates' markup to allow urls like url-of-main-blogview#trackbacks, but not yet :O)

• The Seven Challenges of e-Learning design (Part 2) ( just trackbacks )
• Portfolios, social software and non formal learning ( just trackbacks )

Phew! Trivial task turns into testing trial... I've just changed the layout of links in the Subscribe sidebar, so that the whizzy new RSS-2 filecasting, main-content feeds were exposed to the feed-consuming public.

I also took the opportunity to add a link to a new bit of stub content explaining all the formats and options ( there are loads of undocumented feed sources at the moment, for instance categories). I also flagged the default feed ( the RSS-2 that will be auto- discovered) on the assumption that it would be good to have one simple choice, and also to indicate for users with auto-discovery which of the feeds they will get.

We've been remiss in keeping the documentation of progress flowing into this weblog. Apologies for that - but it's durned hard to do it and write about it at the same time :o) - So I'm going to try re-using content to help spread the word. I posted into the site-editors weblog for the National Guidance Research Forum this morning with a list of pending and in-progress to-dos. Most of these will also become core KNotes improvements, so are worth noting here.

This post is a copy of an email I sent this morning, briefly documenting some small and large jobs we agreed in the meeting in Bangor but which have not yet gotten attended to or are still in progress