Difference between revisions of "Development/Tutorials/Akonadi/SerializerPlugin"

Jump to: navigation, search
m (Text replace - "<code xml>" to "<syntaxhighlight lang="xml">")
Line 83: Line 83:
  
 
The file format or rather serialization format is assumed to be a simple XML based one, basically looking like this:
 
The file format or rather serialization format is assumed to be a simple XML based one, basically looking like this:
<code xml>
+
<syntaxhighlight lang="xml">
 
<history>
 
<history>
 
   <header>
 
   <header>

Revision as of 19:37, 29 June 2011


Contents

Development/Tutorials/Akonadi/SerializerPlugin


Using your own data type with Akonadi
Tutorial Series   Akonadi Tutorial
Previous   C++, Qt, KDE development environment
What's Next  
Further Reading   CMake, Akonadi Development Tools

This tutorial will guide you through the steps of getting your own data type ready for usage with Akonadi.

Akonadi itself has been designed to be able to handle any kind of data, so it needs to be told what kind of data an item is actually holding and how to convert it between programming language data structures and the universal array of bytes (sometimes also referred to as an octet stream).

Once you have support for the basic handling of your data type as an item payload, you can proceed to write a resource for persistant storage of the data. For details on doing that see the Akonadi Resource Tutorial

Prerequisites

noframe
 
This section needs improvements: Please help us to

cleanup confusing sections and fix sections which contain a todo


Describe required versions and build setup

Preparation

To use your own data type with KDE's API for Akonadi, you will need three things:

  • a MIME type for identifying your data
  • a programming language data structure (e.g. class) with value behavior
  • methods or helper classes to write your data into a byte array and read from a byte array

MIME Type

Akonadi uses the MIME type of items to check whether one if its collections can be used as a target for new items carrying data of that MIME type. It is also used to map to a serializer plugin, i.e. a helper class which can create the high level data structures of your data type given a byte array and vice versa.

Depending on what your data type is there might be a standard MIME type, e.g. you are the first to implement Akonadi support for it, or you might already be using one for file associations, etc.

Otherwise you will have to "invent" one, e.g. by using the MIME "namespace" reserved for unofficial types, which start with application/x-. If your application is part of the KDE application suite, you might want to use a MIME type of the following form: application/x-vnd.kde.yourappname

Ktip.png
 
Tip
This becomes important once you ship your Akonadi support. It is the identifier for your data type.


Data structure with value behavior

KDE's implementation of an Akonadi API, or more precisely the Akonadi::Item payload methods, need a high level representation of your data in the form of a class which's instances can be copied and assigned like values.

This can either be done by only having members which have value like behavior themselves, or by using a reference counting approach where instances share a pointer to the actual data.

The latter approach can most easily be achieved by using QSharedDataPointer and QSharedData together with the d-pointer idiom. As an example see how this is being used for the class which represents a KDE address book contact, KABC::Addressee

If a lot of your current API is pointer based and you'd rather not have to reimplement your data classes under the approach described above, you can work around this using an explicit shared pointer such as boost::shared_ptr. As an example see how this is being used for the class which represents a KDE calendar entry, KCal::Incidence: code using instances of this class with Akonadi create a typedef like this

  1. include <boost/shared_ptr.hpp>

typedef boost::shared_ptr<KCal::Incidence> IncidencePtr; and then use this type with the item's payload methods.

Ktip.png
 
Tip
Consider this only as a fallback. Having to remember pointer ownership through various levels of indirection and sharing will make debugging a lot more difficult.


Since your data classes will be used by your application, your Akonadi resource and by the serializer plugin, you will either have to build them as sub projects of a containing master project or put them into a shared library so they can be used by all three targets.

Data Serialization

Unless you are starting with a totally new application, you will already have means to save your data type to a file or something similar.

If you can abstract this to use QIODevice rather an QFile, or have already done so, you can use the same classes or methods to handle the data serialization for Akonadi.

Ktip.png
 
Tip
Consider doing this anyway, since your items will have to be persistenly stored by an Akonadi resource anyway and the most simple form of persistant storage is a local file.


Tutorial Example

In order to demonstrate certain aspects of developing an Akonadi serializer plugin, consider the following example as a starting point similar to your situation.

The data type in this example is the history or chat log of an instant messaging client, i.e. a recording of the text messages between the local user and a single (for the sake of simplicity) remote user.

Data Format

The file format or rather serialization format is assumed to be a simple XML based one, basically looking like this:

<history>
  <header>
    <local id="contactId1" />
    <remote id="contactId2" />
  </header>
  <messages>
    <message who="contactId1" when="2009-01-01T12:00:00">Hello</message>
    <message who="contactId2" when="2009-01-01T12:01:10">Hi</message>
    <message who="contactId1" when="2009-01-01T12:02:04">Video at my place, 20:00?</message>
    <message who="contactId2" when="2009-01-01T12:04:00">Sure, I'll bring popcorn</message>
    <message who="contactId1" when="2009-01-01T12:04:50">Great! See you later</message>
    <message who="contactId2" when="2009-01-01T12:05:40">Yeah, cya!</message>
  </messages>
</history>
</code>
 
=== Data Classes ===
 
The classes for handling the history basically look like this:
 
<code cppqt>
class History
{
public:
  class Message
  {
  public:
    typedef QList<Message> List;
 
    Message();
 
    void setSender( const QString &contactId );
 
    QString sender() const;
 
    void setText( const QString &text );
 
    QString text() const;
 
    void setTimestamp( const QDateTime &timestamp );
 
    QDateTime timestamp() const;
 
  private:
    class Private;
    QSharedDataPointer<Private> d;
  };
 
  History();
 
  void setLocalContactId( const QString &contactId );
 
  QString localContactId() const;
 
  void setRemoteContactId( const QString &contactId );
 
  QString remoteContactId() const;
 
  void addMessage( const Message &message );
 
  Message::List messages() const;
 
  static QString mimeType();
 
private:
  class Private;
  QSharedDataPointer<Private> d;
};
</code>
 
The full version can be downloaded here: [http://people.freedesktop.org/~krake/akonadi/serialzertutorial/history.h history.h], [http://people.freedesktop.org/~krake/akonadi/serialzertutorial/history.cpp history.cpp]
 
Both the main class as well as the nested class use the facilities provided by {{qt|QSharedDataPointer}} and {{qt|QSharedData}} to implement the value like behavior required by [http://api.kde.org/4.x-api/kdepimlibs-apidocs/akonadi/html/classAkonadi_1_1Item.html Akonadi::Item]'s payload methods.
 
=== Data I/O ===
 
The class for reading and writing the XML format look basically like this:
<code cppqt>
class HistoryXmlIo
{
public:
  static bool writeHistoryToXml( const History &history, QIODevice *device );
 
  static bool readHistoryFromXml( QIODevice *device, History &history );
};
</code>
 
The full version can be downloaded here: [http://people.freedesktop.org/~krake/akonadi/serialzertutorial/historyxmlio.h historyxmlio.h], [http://people.freedesktop.org/~krake/akonadi/serialzertutorial/historyxmlio.cpp historyxmlio.cpp]
 
== Getting started ==
 
All serializer plugins are implementations of the [http://api.kde.org/4.x-api/kdepimlibs-apidocs/akonadi/html/classAkonadi_1_1ItemSerializerPlugin.html Akonadi::ItemSerializerPlugin] interface.
 
We can kick-start the serializer plugin by using '''KAppTemplate''', which can be found as '''KDE template generator''' in the development section of the K-menu, or by running '''kapptemplate''' in a terminal window.
 
{{note|This template is only available with KDE SC 4.3 and later}}
 
First, we select the '''Akonadi Serializer Template''' in the C++ section of the program, give our project a name and continue through the following pages to complete the template creation.
 
[[Image:KAppTemplateAkonadiSerializer.png]]
 
A look at the generated project directory shows us the following files:
<code>
akonadi-serializer.png
akonadi_serializer_imhistory.desktop
CMakeLists.txt
README
akonadi_serializer_imhistory.cpp
akonadi_serializer_imhistory.h
</code>
 
At this stage it is already possible to compile the plugin, so we can already check if our development environment is setup correctly by creating the build directory and having CMake either generate Makefiles or a KDevelop project file.
 
=== Generating Makefiles ===
 
From within the generated source directory:
<code>
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=debugfull ..
</code>
and run the build using make as usual.
 
=== Generating a KDevelop project file ===
 
From within the generated source directory:
<code>
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=debugfull -G KDevelop3 ..
</code>
and open the generated project with KDevelop and run the build process from there.
 
=== Adjusting the plugin description file ===
 
The capabilities of the serializer plugin need to be described in both human understable and machine interpretable form.
This is achieved through a so-called desktop file, similar to those installed by applications.
 
Since the '''akonadi_serializer_imhistory.desktop''' file generated by KAppTemplate contains only example values, we need to edit it:
 
<code ini>
[Misc]
Name=Instant Messaging History Serializer
Comment=An Akonadi serializer plugin for Instant Messaging History data
 
[Plugin]
Type=application/x-vnd.kde.imhistory
X-KDE-Library=akonadi_serializer_imhistory
</code>
 
'''Name''' and '''Comment''' strings might be visible to the user and can be translated. Since our plugin works on our instant messaging data, the MIME type field '''Type''' needs to set accordingly.
 
=== Adding the data specific classes to the build ===
 
As already mentioned in the [[#Data Serialization|Data Serialization]] section, the usual way to have the code needed for data processing shared between the application, the resource, and the serializer plugin is to put the respective classes into a shared library.
 
However, for the purpose of this tutorial we are just going to compile them into the plugin by adding them to the '''CMakeLists.txt''' file by changing
<code>
set( akonadi_serializer_imhistory_SRCS
  akonadi_serializer_imhistory.cpp
)
</code>
 
to this
 
<code>
set( akonadi_serializer_imhistory_SRCS
  akonadi_serializer_imhistory.cpp
  history.cpp
  historyxmlio.cpp
)
</code>
 
The files can be downloaded here: [http://people.freedesktop.org/~krake/akonadi/serialzertutorial/history.h history.h], [http://people.freedesktop.org/~krake/akonadi/serialzertutorial/history.cpp history.cpp], [http://people.freedesktop.org/~krake/akonadi/serialzertutorial/historyxmlio.h historyxmlio.h], [http://people.freedesktop.org/~krake/akonadi/serialzertutorial/historyxmlio.cpp historyxmlio.cpp]
 
== Full Payload ==
 
The most simple way of serialization is to always transfer the whole data between the in-memory data structure and the external format.
This is usually also the method already implemented in data I/O classes, therefore the example XML I/O code implements this behavior as well.
 
First we need a couple of new include directives:
<code cppqt>
#include "history.h"
#include "historyxmlio.h"
 
#include <akonadi/item.h>
</code>
 
Then we implement the '''serialize''' method like this:
<code cppqt>
void SerializerPluginIMHistory::serialize( const Item& item, const QByteArray& label, QIODevice& data, int &version )
{
  Q_UNUSED( version );
 
  if ( label != Item::FullPayload || !item.hasPayload<History>() )
    return;
 
  const History history = item.payload<History>();
 
  HistoryXmlIo::writeHistoryToXml( history, &data );
}
</code>
 
and the '''deserialize''' method like this:
<code cppqt>
bool SerializerPluginIMHistory::deserialize( Item& item, const QByteArray& label, QIODevice& data, int version )
{
  Q_UNUSED( version );
 
  if ( label != Item::FullPayload )
    return false;
 
  History history;
  if ( !HistoryXmlIo::readHistoryFromXml( &data, history ) )
    return false;
 
  item.setPayload<History>( history );
 
  return true;
}
</code>
 
Given existing data I/O facilities, doing full payload serializations is rather trivial and totally sufficient for most data types.
 
== Partial Serialization ==
 
In cases where it is possible to split the data into identifyable parts, serializing such parts independently can be used to reduce the amount of data to be transferred between Akonadi and its clients.
For example only getting the data necessary to populate a list view and getting the rest once the user indicates interest in a specific item.
 
In the case of the example used in this tutorial, the header identifying the communication partners and the list of messages are such identifyable and separately transmittable parts.
 
Lets first implement this in the XML I/O helper class.
 
In file '''history.h''' we add to identifier contants:
<code cppqt>
static const char *HeaderPayload;
static const char* MessageListPayload;
</code>
 
and define them in file '''history.cpp''':
<code cppqt>
const char *History::HeaderPayload = "Header";
const char *History::MessageListPayload = "MessageList";
</code>
 
In the file '''historyxmlio.h''' we add two method pairs for reading and writing the individual parts.
<code cppqt>
static bool writeHistoryHeaderToXml( const History &history, QIODevice *device );
 
static bool readHistoryHeaderFromXml( QIODevice *device, History &history );
 
static bool writeHistoryMessagesToXml( const History &history, QIODevice *device );
 
static bool readHistoryMessagesFromXml( QIODevice *device, History &history );
</code>
 
In file '''historyxmlio.cpp''' we implement them as follows:
<code cppqt>
bool HistoryXmlIo::writeHistoryHeaderToXml( const History &history, QIODevice *device )
{
  if ( device == 0 || !device->isWritable() )
    return false;
 
  QXmlStreamWriter writer( device );
  writer.setAutoFormatting( true );
 
  writer.writeStartDocument();
 
  writeHeader( history, writer );
 
  writer.writeEndDocument();
 
  return true;
}
 
bool HistoryXmlIo::readHistoryHeaderFromXml( QIODevice *device, History &history )
{
  if ( device == 0 || !device->isReadable() )
    return false;
 
  QXmlStreamReader reader( device );
 
  History partialHistory;
  if ( !readHeader( reader, partialHistory ) )
    return false;
 
  // only overwrite header fields
  history.setLocalContactId( partialHistory.localContactId() );
  history.setRemoteContactId( partialHistory.remoteContactId() );
 
  return true;
}
 
bool HistoryXmlIo::writeHistoryMessagesToXml( const History &history, QIODevice *device )
{
  if ( device == 0 || !device->isWritable() )
    return false;
 
  QXmlStreamWriter writer( device );
  writer.setAutoFormatting( true );
 
  writer.writeStartDocument();
 
  writeMessages( history, writer );
 
  writer.writeEndDocument();
 
  return true;
}
 
bool HistoryXmlIo::readHistoryMessagesFromXml( QIODevice *device, History &history )
{
  if ( device == 0 || !device->isReadable() )
    return false;
 
  QXmlStreamReader reader( device );
 
  History partialHistory;
  if ( !readMessages( reader, partialHistory ) )
    return false;
 
  // save header fields and then overwrite external instance
  partialHistory.setLocalContactId( history.localContactId() );
  partialHistory.setRemoteContactId( history.remoteContactId() );
 
  history = partialHistory;
 
  return true;
}
</code>
 
The plugin's '''serialize''' method changes to this:
<code cppqt>
void SerializerPluginIMHistory::serialize( const Item& item, const QByteArray& label, QIODevice& data, int &version )
{
  Q_UNUSED( version );
 
  if ( !item.hasPayload<History>() )
    return;
 
  const History history = item.payload<History>();
 
  if ( label == Item::FullPayload ) {
    HistoryXmlIo::writeHistoryToXml( history, &data );
  } else if ( label == History::HeaderPayload ) {
    HistoryXmlIo::writeHistoryHeaderToXml( history, &data );
  } else if ( label == History::MessageListPayload ) {
    HistoryXmlIo::writeHistoryMessagesToXml( history, &data );
  }
}
</code>
 
and the '''deserialize''' method to this:
<code cppqt>
bool SerializerPluginIMHistory::deserialize( Item& item, const QByteArray& label, QIODevice& data, int version )
{
  Q_UNUSED( version );
 
  // readHistoryHeaderFromXml() and readHistoryMessagesFromXml() only get parts of a full history item.
  // therefore make sure that the eventually already fetched other part is not lost by initializing
  // the local instance with the current payload if it exists.
  History history;
  if ( item.hasPayload<History>() )
    history = item.payload<History>();
 
  if ( label == Item::FullPayload ) {
    if ( !HistoryXmlIo::readHistoryFromXml( &data, history ) )
      return false;
  } else if ( label == History::HeaderPayload ) {
    if ( !HistoryXmlIo::readHistoryHeaderFromXml( &data, history ) )
      return false;
  } else if ( label == History::MessageListPayload ) {
    if ( !HistoryXmlIo::readHistoryMessagesFromXml( &data, history ) )
      return false;
  } else
    return false;
 
  item.setPayload<History>( history );
 
  return true;
}
</code>
 
and the '''parts''' method like this:
<code cppqt>
QSet<QByteArray> SerializerPluginIMHistory::parts( const Item &item ) const
{
  QSet<QByteArray> partIdentifiers;
 
  if ( item.hasPayload<History>() ) {
    const History history = item.payload<History>();
 
    bool hasHeader = false;
    bool hasMessageList = false;
 
    if ( !history.localContactId().isEmpty() && !history.remoteContactId().isEmpty() ) {
      partIdentifiers << History::HeaderPayload;
      hasHeader = true;
    }
 
    if ( !history.messages().isEmpty() ) {
      partIdentifiers << History::MessageListPayload;
      hasMessageList = true;
    }
 
    if ( hasHeader && hasMessageList )
      partIdentifiers << Item::FullPayload;
  }
 
  return partIdentifiers;
}
</code>
 
The two identifiers '''History::HeaderPayload''' and '''History::MessageListPayload''' can now be used with [http://api.kde.org/4.x-api/kdepimlibs-apidocs/akonadi/html/classAkonadi_1_1ItemFetchScope.html Akonadi::ItemFetchScope] to only get the respective portion of the data.
 
For example in combination with an [http://api.kde.org/4.x-api/kdepimlibs-apidocs/akonadi/html/classAkonadi_1_1ItemModel.html Akonadi::ItemModel] to just show all available conversations without loading all their messages:
<code cppqt>
Akonadi::ItemModel *model = new Akonadi::ItemModel();
model->fetchScope().fetchPayloadPart( History::HeaderPayload );
</code>
 
== Reference Implementation ==
 
The files used for the tutorials reference implementation can be found here: [http://people.freedesktop.org/~krake/akonadi/serialzertutorial/akonadi_serializer_imhistory.h akonadi_serizalizer_imhistory.h] and [http://people.freedesktop.org/~krake/akonadi/serialzertutorial/akonadi_serializer_imhistory.cpp akonadi_serizalizer_imhistory.cpp]
 
 
[[Category:Tutorial]]
[[Category:C++]]
[[Category:KDE4]]
[[Category:Akonadi]]
[[Category:PIM]]

KDE® and the K Desktop Environment® logo are registered trademarks of KDE e.V.Legal