Difference between revisions of "Projects/PIM/Akonadi"

Revision as of 08:49, 4 February 2009

Akonadi TODO

The following list contains the things which need to be done for Akonadi.

Note: The person noted in the "Contact" column is not necessarily the one implementing that feature, but the one who can tell you what to do and how to help, i.e. you can also contribute to those tasks ;-)

Core

KDE 4.1 / Akonadi 1.0

Urgent tasks that need to be finished for the KDE 4.1 release (May 19th):

KDE 4.2 / Akonadi 1.1

That's the stuff we want to have in 4.2, very roughly ordered by priority. Releases are scheduled for early January 2009.

Scheduled for KDE 4.3 / Akonadi 1.2

Stuff we want to do for KDE 4.3, scheduled to be released in XXX 2009. The list is not completed yet, will be discussed during the Osnabrueck 7 meeting.

Post KDE 4.3 / Akonadi 1.2

This stuff is currently not considered for 4.3 due to lack of resources. Of course it will be added nevertheless if someone implements it. The list is completely unsorted.

Supported Types

Overview on the current state of support for various types:

(1) see Email

Mail specific extensions

Event/Todo/Journal specific extensions

Resources, Agents and others

Resource status overview (this should list all resources existing in KDE3 or already under development for Akonadi):

1. only full sync supported currently, need optimization
2. does not yet honor cache policy
3. still relies on QSettings for configuration and/or doesn't provide configuration over D-Bus
4. does not yet provide correct access control settings
5. only adding new items, not changing existing ones
6. availability of child collections depend on whether the KResource plugin has subresources
7. child collections can be added or removed if the KCal plugin can have subresources

KResource Migration Status

Everything without migration support is implicitly converted to use the compat bridge currently.

KABC migration status

KCal Migration Status

Akonadi Braindump

Ideas/notes etc. on various open issues in Akonadi.

Akonadi Standard Actions

Idea: Have something like KStandardAction for Akonadi that not only includes the representation of the action but also its state management and the actual operations. Like libakonadi that should be splitted into a generic, type-independent part and be extensible for type-specific actions. This will enable code sharing among many applications and guarantee consistent actions everywhere.

State management: watch selection models of a collection and/or item model.

Use KXMLGUI for context menus in standard views to allow easy extensibility with custom actions.

Generic actions:

• new collection [done]
• new virtual collection
• delete collection [done for single selection]
• copy collection [done]
• cut collection
• paste collection [done]
• synchronize collection [done for single selection]
• synchronize resource
• synchronize everything
• show collection properties dialog [done]
• delete item(s) [done]
• copy item(s) [done]
• cut item(s)
• paste item(s) [done]
• paste native data
• tag item(s)
• comment item
• rate item(s)
• configure resource
• add resource? (would need a type parameter, etc.)
• delete resource
• manage local subscriptions [done]
• move to submenu
• copy to submenu

The list is definitely long enough to make this worthwhile.

Collection Model / Collection View

Ideas/missing features/bugs of the collection model/view:

• Enable/disable status columns
• Show status after the name (see KMail)
• Size column
• Save/restore layout
• Custom collection icons (see KMail, also needed for resource defined special folders (eg. Inbox)
• Status tooltips (see KMail in 3.5.9)
• Quick search
• Favorite folder view as proxy model on top of the normal collection model (FlatCollectionProxyModel might be helpful for there)
• Accumulated statistics (unread, total, size)
• Fix dnd: move/copy/cancel menu always shows up and never has any effect

Compatibility

Notes on how to keep long-term protocol, source and binary compatibility.

• Detect server version in Akonadi::Session, might be useful in case of protocol extensions/changes
• What about database server version updates?
• Versioning or any other kind of serialization format meta data, we'll need that in case of changes in serialization formats (see eg. Robert's compression patch where this is needed)
• We currently use 32 bit ids, probably hard to change later on, is that enough?
• Plugin versioning

Resource API issues

Notes/ideas/complaints about the Akonadi::ResourceBase API:

• Extremely error prone:
  • Scheduler dead-locks when a requested operation is not correctly announced as finished, esp. a problem in error cases.
  • Using the result methods multiple times or when not requested asserts
• Item streaming is missing, requiring all data to be in memory at once
• Non-incremental updates need to know the results of ItemSync to not have to provide all data, even for already existing/unchanged items.

API / BC issues

Smaller stuff that should be fixed before the ABI freeze and is not yet listed above:

• Cleanup the D-Bus format used by NotificationMessage
• No mentioning of "IMAP" in the public API, we are not using IMAP
• Naming and installed location of libraries, headers and executables (see discussion on kde-pim ML)
• Payload part labels are QStrings, attribute types are QByteArray

Deployment issues

• Multiple access: Should multiple Akonadi instances' mysqlds access a single set of data files the mysql will likely corrupt the data. This can happen in any NFS+YP installation where users can log onto any machine and access shared homes. MySQL relies on filesystem locking to prevent multiple access. MySQL multiple instance docu.
• InnoDB tables should not be used on NFS.
• NFS speed: MySQL documentation recommends against locating its data files on network shares.
• AppArmor: Distros' AppArmor profiles prevent MySQL from writing outside its defined data directory (usually /var/lib/mysql). This is a problem at least with *buntu and openSUSE. These will need to be adapted. It is possible to daisy-chain profiles so that MySQL started by Akonadi receives a different profile to MySQL running standalone. An empty profile has all rights. Another possibility is to adapt the general MySQL profile so it can write to ${XDG_DATA_HOME}/akonadi(usually ${HOME/}.local/share/akonadi). Both support for profile chaining and ${HOME} may depend on the version of AppArmor installed. What about SELinux?
• Backup: MySQL data files should not be backed up without telling the mysqld process, otherwise a corrupt backup will be made. LOCK TABLES and FLUSH TABLES at the least. A dump can be made or mysqlhotbackup may be used in some circumstances. We should consider sysadmins' backup techniques when planning/promoting Akonadi, as a simple rsync cronjob with running Akonadi will not work. MySQL backup advice.

Account Creation Wizard

Notes on the resource configuration wizard discussion.

• WizardBase class
• Wizard discovery via .desktop files
• A wizard can configure multiple resources (eg. IMAP + LDAP for Kolab) and non-resources (mailtransport)
• Optionally hide resources completely covered by wizards in the Add Account dialog
• List wizards in the Add Account dialog
• Store Account -> Resources information to offer deletion of depending resources

PostgreSQL datastore support

First review of the akonadi code reveals PGSQL Qt driver is already supported, but the code is not totally ready for it. Some first reading comments about where to patch for supporting PostgreSQL too:

akonadi/server/src/storage/dbinitializer.cpp

• line 201..., ALTER TABLE ADD [UNIQUE] INDEX syntax is MySQL only, better use the common one CREATE [UNIQUE] INDEX ... one [done with revision 789899]
• line 303, QString DbInitializer::sqlType(const QString & type), LONGBLOB is the MySQL type, PostgreSQL will use BYTEA.
• line 362, qFatal("Implement index support for your database!");

 mysql: SHOW INDEXES FROM %1
 pgsql: SELECT c2.relname, i.indisprimary, i.indisunique, i.indisclustered, i.indisvalid, pg_catalog.pg_get_indexdef(i.indexrelid, 0, true), c2.reltablespace
 FROM pg_catalog.pg_class c, pg_catalog.pg_class c2, pg_catalog.pg_index i
 WHERE c.oid = '16713' AND c.oid = i.indrelid AND i.indexrelid = c2.oid
 ORDER BY i.indisprimary DESC, i.indisunique DESC, c2.relname
 
    relname    | indisprimary | indisunique | indisclustered | indisvalid |                       pg_get_indexdef                        | reltablespace
 --------------+--------------+-------------+----------------+------------+--------------------------------------------------------------+---------------
  foo_pkey     | t            | t           | f              | t          | CREATE UNIQUE INDEX foo_pkey ON foo USING btree (id)         |             0
  foo_time_key | f            | t           | f              | t          | CREATE UNIQUE INDEX foo_time_key ON foo USING btree ("time") |             0
 (2 lignes)

akonadi/server/src/storage/datastore.cpp

• line 700, qint64 DataStore::highestPimItemId() const, seems wrong for a concurrent program (SELECT MAX(col) FROM table; is not reliable if another thread is inserting at the same time) : check the callers
• line 794, qint64 DataStore::lastInsertId( const QSqlQuery & query ), PostgreSQL doesn't provide concurrent-bogus last_insert_id, but from 8.2 accepts the syntax INSERT INTO ... RETURNING col1, col2, ...;
• line 806, QString DataStore::dateTimeFromQDateTime( const QDateTime & dateTime ), use PostgreSQL to_char() formatting function for date rather than having Qt do it

Documentation

• Avoid examples using KJob::exec() due to its problematic side-effects.
• Examples in the ItemCreateJob docs (and maybe others) operate on items in the root collection which does not work at all
• Resurrect the server API docs
• Content of http://api.kde.org/4.x-api/kdepim-apidocs/akonadi/html/index.html should probably be moved to server or client lib docs

Migration of pre-Akonadi data and settings

• KCal/KABC
  • switch KRes resources to Akonadi KRes compat resources, use KRes Akonadi compat resource, basically injecting Akonadi inbetween
  • Replace Akonadi KRes compat resources with their ported equivalent
  • Application side does not need migration code, "only" port to Akonadi
• KMail
  • Should migrate to one resource per account + one for previous local folders
  • Problem: local folders are mixed mbox/maildir
    • Dedicated KMail compat resource?
    • Convert everything to maildir first?
  • Flags are stored in proprietary binary format
• KNode
  • Migrate every account to NNTP resource
  • Local flags should be preserved, stored in proprietary binary format
  • Local subscription state should be preserved (where is that stored?)
  • Local folders using MBOX format + proprietary binary index
• Akregator

Search

• Delegation to resources still completely missing
  • Requires query transformation
  • Requires management for live searches
  • Requires the ability to report search results (needs protocol extension)
• Query language still undecided, current options: XESAM, SPARQL
• Backend still undecided, current options: XESAM, Nepomuk

KMail Breakdown Plan

The current plan is to put some parts of KMail into a stand-alone library, independent of KMail. This increases code reuse (for example, the message composer could be shared with Mailody) and makes the code a lot easier to maintain and to port to Akonadi.

The Road to World Domination

Sort of related so the stuff above but with a scope for all of KDE PIM and beyond.

Akonadi FAQ

This FAQ primarily deals with technical and design questions, if you have questions about using Akonadi (such as "I get error X when starting Akonadi"), please refer to userbase.kde.org/Akonadi.

Where do I find the Akonadi config files?

~/.config/akonadi/

Where does Akonadi store my data?

~/.local/share/akonadi/

Where can I find the source code?

There are three different parts of source code:

• The Akonadi server, which is KDE-independent. Currently, the code can be found at kdesupport/akonadi. The server code will likely move to the freedesktop.org project in the future, the move has already been requested.
• The Akonadi KDE client libraries are in kdepimlibs/akonadi.
• Various agents, resources, the Akonadi console and much other misc stuff is in kdepim/akonadi

Where can I find documentation?

The documentation is mainly in the code itself, in the form of doxygen comments. You can find the generated documentation on [api.kde.org api.kde.org], for example:

• General documentation, including a design overview, is here

Note that some links in the above documentation which point to the server or the KDE client libraries are currently broken, use the links below to access that documentation.

• Documentation for the KDE Akonadi client library is here
• Documentation about the server is here

Which DBMS does Akonadi use?

So far only MySQL. There is some work on PostgreSQL support going on though. Basically, every database that is supported by QtSQL can be used, requiring minimal changes in the code at most. However, not all of them provide the features needed by Akonadi (see next two questions).

Why not use sqlite?

We tried. Really. It can't handle the concurrent access very well, in the best case this means very slow operations, but we've also seen deadlocks and failing transactions. Once that's fixed in sqlite, adjusting Akonadi to use it again instead of MySQL is no problem.

Why not use MySQL/Embedded?

We tried that as well, there are two reasons for not using it: No support for the InnoDB engine (which we need for transaction support) and poor availability (only OpenSUSE provided usable packages, needed a patched QSQL driver).

Do I need a running MySQL server?

No. Akonadi starts its own local MySQL server (unless configured otherwise, see next question). All you need is having the 'mysqld' binary installed at runtime (usually found in the mysql-server package of your distribution).

Can Akonadi use a normal MySQL server running on my system?

Yes, it can. You find the corresponding settings in ~/.config/akonadi/akonadiserverrc.

I don't like a database server because of backups/running on NFS/etc.

See section "Deployment Issues" above, we are aware of that and working on it. Some of these, like backup/restore are already implemented. But please be aware that most of this issues also existed before (eg. with KMail's custom binary indexes).

Can a single Akonadi instance be used by multiple users?

No. There has to be one Akonadi server instance per user. However, it is possible to use a shared database server.

Can I access the Akonadi server on a remote machine?

No. Akonadi is not a groupware server. It's a local cache only.

What's the differences between Akonadi and EDS?

EDS is limited to contacts and calendar data, Akonadi is type-independent. Especially, Akonadi is designed to also handle high-volume data such as email. Akonadi and EDS also differ largely in their access protocol on a technical level (Corba/D-Bus vs. local socket with IMAP-like protocol/D-Bus) and also on a protocol level (type specific vs. generic).

How do I create a collection?

From the developers point of view, there is Akonadi::CollectionCreateJob for that, from the users point of view, most applications allow that, eg. Mailody or akonadiconsole.

However, there is one limitation: Top-level collections can only be created by resources, not by normal applications. The reason for that is that every object (collection or item) is supposed to have an owning resource, that is a resource that is responsible for storing and retrieving that object. There is an ongoing discussion to remove this restriction though.

So, if you want to create a collection eg. for testing purposes, add a resource first. Most Akonadi applications offer an option to do that, a module for KControl is planned as well.

How do I disable automatic migration from KDE's traditional framework?

The migration tool is controlled by standard KDE configuration file called kres-migratorrc.

Distributors or system administrators wanting to disable the automatism will probably want to that globally, e.g. by editing the installed default configuration file, or by using KDE's configuration hierachy and using a profile config between that and the user level.

The quickest way to deactivate it for one user account only is to use KDE's kwriteconfig tool to set the respective configration value with a simple copy&paste of the following command:

kwriteconfig --file kres-migratorrc --group Migration --key Enabled --type bool false

Please note that at some point KDE applications such as Kontact, KOrganizer, KMail, KAddressBook will be using Akonadi directly, at which point migration either has to be enabled or performed manually.

How do I completely disable Akonadi startup?

If you already have applications natively using Akonadi, you of course can't disable Akonadi startup. Mailody and KPilot are applications that are already based on Akonadi, for example.

Other applications, like KAddressbook and KOrganizer, are not based on Akonadi yet, at the time of writing. Instead, they use the old KResource framework for storing contacts, calendars and notes. During the KDE 4.2 beta time, these KResources were automatically migrated to Akonadi-based KResources, the so-called Akonadi compatibility resources. Therefore, applications like KAddressbook and KOrganizer would use Akonadi indirectly through KResources, and therefore would start the Akonadi server when being started. Note that the same situation applies for KMail, which uses addressbook functionality also based on KResources.

If Akonadi doesn't start up correctly for you, the following should help you to disable Akonadi startup and use your old KResources again.

First, disable automatic migration like described in the above FAQ entry. Then, open System Settings, go to the Advanced tab and open the KDE Resources config panel. There, you can configure which type of KResources are used for contacts, calendars and notes. If the migration to Akonadi was successful, you'll probably only see the Akonadi Compatibility Resource as an active resource, and all others disabled.

To disable Akonadi startup, enable your old resources again, then disable and delete the Akonadi compatibility resource.

What is the relation between Akonadi and OpenSync?

Akonadi and OpenSync focus on different aspects and complement each other. Akonadi provides a unified way to access PIM data for applications and a framework to implement powerful connectors to varies data sources. OpenSync focus is on syncing two sets of PIM data.

An Akonadi plugin for OpenSync is currently under development, allowing to sync PIM data available through Akonadi with any other system supported by OpenSync, especially mobile phones.

Links

• Akonadi Testing Framework
• Akonadi Development Tools
• Tutorials:
  • Development/Tutorials/Akonadi/Application
  • Resource Development
  • Serializer Plugin Development
• Bookmark support based on Akonadi
• http://pim.kde.org/akonadi/
• #akonadi IRC channel on freenode
• Meetings
  • Halloween Sprint 2008
  • Osnabrueck 7