Projects/PIM/Roadmap

< Projects‎ | PIM
Revision as of 18:07, 12 June 2014 by Cmollekopf (Talk | contribs)

Jump to: navigation, search

This page is meant to collect various planned features/enhancements. For the release planning all features that you want to implement should be listed in http://techbase.kde.org/Schedules/KDE4/4.13_Feature_Plan (you can for instance just link here).

Note that this page is strictly for developers, and NOT a wishlist.

Contents

Roadmap / Planned Features

This section is for features where you have concrete plans for implementation (doesn't have to be in the next release). A short description, the approximate timeline if you know already (e.g. for version X), and the reason why the feature is needed would be interesting.

New Kolab-Resource that derives from the IMAP-Resource

  • By: Christian Mollekopf
  • Planned for: 4.14

I plan to replace the kolab proxy with a new resource that derives from the imap resource (so no additional agent anymore). Each imap resource would have two root collections, one for email and one for groupware data.

Rationale

The primary reason is to reduce the overall complexity of the design, I expect the new resource to be easier to debug (because it's mostly the same codebase as the imap resource) and to be less error prone (because it's one less level of indirection). Other reasons are:

  • improved performance: we can convert the objects directly inline, which saves processing power.
  • we have one configuration per account (as opposed to the kolabproxy), because each resource is only responsible for one account, allowing me to get rid of the per account configuration hacks in the kolabproxy.
  • A kolab account no longer requires two configuration dialogs, and the dialog can probably be simplified for kolab. (at least by default we only require server + login + password).
  • The groupware folders no longer need to be hidden in the email client because they are not created anymore

Range Queries

  • By: Christian Mollekopf
  • Planned for: 4.14

It should be possible to request items by certain properties, such as a start and end date of an event, so not all items need to be loaded into memory.

Rationale

One of the largest performance limitations is that we currently have to load all items into memory in order to be able to select the ones we're interested in, since the akonadi server doesn't know the payload. We therefore require indexes of certain interesting properties that allow us to query by these properties.

Implementation

The problem is very similar to search, so it can probably be solved similarly. I'm not sure whether the indexing should happen in the baloo indexer, or in a preprocessor, since IMO fulltext indexing can be "eventually consistent" (5min delay wouldn't be a big problem), where these indexes are required to make the item available to the application (the event will not appear before the corresponding index and open query is updated). Applications would "search" for the item-set, that would be exposed as virtual collection. So the calendar would i.e. search for "events of week 42", for which it would receive an id of a virtual collection that it can monitor. That will give the calendar only the event's it requires for display, including notifications for the range it's interested in only. As the user scrolls in the view, the range query needs to be modified and updated accordingly.

Random notes:

  • I assume incremental updates will be crucial for performing well with large datasets (for the scrolling).
  • The calendar should probably load the whole month if it's displaying only a week to have a buffer for the scrolling.

Relations

  • By: Christian Mollekopf
  • Planned for: 4.14

We want to be able to create relations between items in akonadi, so we can for instance relate a note to an email.

Rationale

A generic mechanism is needed to associate items, other than the existing tags (which are visible and have a name, etc.), and collections (similar to tags). The relations mechanism should provide 1:1 relations with a type. Usecases: note-email, todo-email, ... (full mesh).

Implementation
  • Relation is triple (id, type, id)
  • relation types are stored in RelationTypeTable
  • RelationCreateJob( id, type, id)
  • ReleationDeleteJob (id, type, id)
  • ReleationFetchJob(id, type) => list of items
  • ItemFetchJob - include list of items IDs that are in releation of specified type(s) with the fetched item
  • specify type(s) in ItemFetchScope?
  • Resources should be notified as well, so relations can be stored on the server. However, a relation should only be added to the resource if both ends of the relation are part of the resource, and it should be removed again if one end of the relation is not in the resource.

Frameworks

Transition to frameworks, kdepimlibs first, rest will follow later. See http://community.kde.org/Frameworks/Akonadi

Akonadi IDLE

  • By: Dan Vrátil
  • Planned for: 4.14 / Akonadi 1.13

I want to introduce a new method of delivering change notifications from Akonadi server to clients inspired by IMAP IDLE extension.

Rationale

Even with batch notifications and server-side notification filtering, Akonadi is still flooding DBus with large amount of change notifications. When a notifications is delivered to clients' Akonadi::Monitors, they need to query Akonadi for the actual item, which generates additional IO/CPU load and slows down the process.

Implementation

Instead of DBus notifications, each Akonadi::Monitor would open a dedicated session, called IDLE session and push it's notification filter configuration to the server. The Akonadi server would then send change notifications via this session instead of using DBus messages. Unlike with DBus notifications, when additional roundtrip from Monitor to Server is needed to request the actual changed Item, Akonadi IDLE notification will, in addition to change description, already contain the actual Item.

This will make a use of the fact, that Server knows and has the Item (or Collection) at the moment when notification is generated, so it can easily serialize it (once) and send it to all interested parties.

I expect notable performance improvements and decrease in database load. The protocol-based "notifications: are also easier to extend, unlike DBus messages, which would solve our problem with having to introduce a new NotificationMessageVX every time we need to add something.

Server-side Change Recording

  • By: Dan Vrátil
  • Planned for: 4.14 / Akonadi 1.13

I want to move change recording from Agents to the Akonadi server. This feature depends on Akonadi IDLE and paves the path for addition features, like on-demand start/stop of Agents.

Rationale

Currently each Agent records change notifications into it's own file. This duplicates some data (not much, but still) on the disk, but also generates a lot of IO, because we need to sync() it every time, even when the notification stays there just for a brief moment. I propose we store the change notifications on the Akonadi server, which already has a storage suitable for them, but also sees the "bigger picture" of all Agents available and can optimize accordingly.

Additionally, with server-side change recording, we can shut down an Agent when it's not used or usable (think for instance IMAP resource when network connection is not available) and still record all it's notifications on the server, then dispatch them once the Agent is usable again. This on-demand start/stop of Agents however, is a different task/topic, which will probably not be implemented in time for 4.14 (probably a KF5 topic)

Implementation

Implementation of server-side change recording builds on top of Akonadi IDLE. The IDLE command will be extended to provide means for Akonadi::ChangeRecorder to enable/disable/suspend/resume change recording as well as request pending notifications from the server.

How the notifications will be stored on the Akonadi server has yet to be determined.

Resource managment for events

  • By: Sandro Knauß
  • Planned for: 4.14

I want to add a new tab in the event create/edit dialog to add resources to an event like cars,beamber,rooms etc.

Rationale

Currently kmail only knows about other persons as attendees for a event, but for many events it is nessesary to book also rooms, cars. etc. This should be done with another tab in the create/edit dialog of events. The first impementation would only support kolab servers, but should also usable with a own LDAP Server and a service for freebusy files.

A new calendar selection for KOrganizer

  • By: Christian Mollekopf
  • Planned for: 4.14

I plan to replace the current calendar/folder selection widget with a new design that scales better with a lot of folders.

Rationale

The current calendar selection quickly gets very complex with many calendars, and looking at a single calendar involves disabling all currently selected calendars, and enabling the one you want to see, which doesn't scale.

The new version addresses this in three steps:

  • Only a selection of calendar folders is permanently visible (favorites).
  • new calendars can be added permanently or temporarily to that selection
  • A quickview allows to peek at a single calendar easily, without disrupting the regular view.

To additionally unclutter the view, the selection will display all shared calendars of a person under a "Person"-node in the treeview. This way you only get one entry per person instead of one per folder by default, while still having access to the individual calendars when expanding the node.

Synchronize the server-side and local subscriptions

  • By: Christian Mollekopf
  • Planned for: 4.14

The imap and kolab resources should synchronize the server-side subscription state with the local subscriptions, and synchronize all folders by default.

Rationale

Currently, the imap resource filters the retrieved mailbox-list by the subscribed folders. All these folders are then subscribed to locally by default, with the additional option of unsubscribing some of them.

I want to change this to downloading all mailboxes, and synchronizing the server-side subscriptions with the local subscriptions (if server-side subscriptions are enabled). The subscription state would then also be considered for downloading data, so a full sync of a disconnected imap account would only download the data the subscribed folders.

The important advantage is that we can use the unsubscribed, but locally available folders to temporarly download data. That way you can have offline-imap folders, while still being able to i.e. access shared folders as online-imap. It also makes it fairly easy to use the local folder list for autocompletion.

In order to implement this I will need to make the Akonadi::SubscriptionJob public API.

Temporary subscription of collections (referenced collections)

  • By: Christian Mollekopf
  • Planned for: 4.14

A mechanism to temporarily include an otherwise disabled collection in a single application.

Rationale

For the new calendar selection I need to be able to temporarily include otherwise disabled collections. This should make them available from within the application only (not from others), and the collection should be removed again once it is no longer referenced.

This feature will allow for "on-demand" loading of collections, so it is i.e. possible to temporarily look at an archive email folder without including it in the synchronization.

As a first usecase this will be used to temporarily load calendars in the new calendar selection.

Brainstorming

List in this section ideas that you think should be implemented, but you currently have no plans of working on in the near future. It will hopefully allow us to share some of the ideas that are currently only existing in one or the other head.

Job-based Resource interface

  • By: Dan Vratil, Christian Mollekopf

The idea is to replace the current ResourceBase interface with a job-based one. Each task (retrieveItems, itemChanged, collectionAdded, ....), is a job, that can be implemented by each resource implementation.

Rationale

This will result in a simpler interface (since the resources are doing an async operation anyways), make it easier to support parallellism of tasks, etc.

Other benefits are:

  • no more AgentBase::Observer - we can add new new "tasks" withouth having to create another ObserverV42
  • possible paralellization - multiple jobs can be running at the same time, speeding up sync for instance
  • much nicer and easier-to-use API
  • no more asserts because resource implementation called taskDone() when no task was running, etc....
Planned implementation
  • we introduce this as ResourceBaseV2 and deprecate ResourceBase
  • resources are slowly ported to ResourceBaseV2
  • initially the current ResourceScheduler can be used without almost any modifications
  • in time we can introduce pararellization of tasks
  • base jobs: RetrieveItemsJob -> resources implementation reimplement the jobs to provide the actual functionality
  • implementations are registered to resource base ( registerJob<ImapRetrieveEmailsJob>() )
    • Or the resource injects a ResourceTaskFactory that creates the jobs, or would that again result in the ObserverV42 problem?
  • ResourceBase starts the Job like if it was dispatching a task
Testability

While creating ResourceBaseV2 we should also make it testable. See the IMAP-Resource how this could work. Ideally all akonadi related jobs are part of the respective job/task interface, which makes it possible for the tests to test each individual job without requiring an akonadi server.

Calendar events envelopes

  • By: Christian Mollekopf

Extract and store certain data from events into PLD:ENVELOPE part. Allows MemoryCalendar to retrieval all ENVELOPEs and then fetch PLD:RFC822 only for events that are relevant to required data/time.

It first needs to be verified how large the impact of this would be, since calendar data is relatively small (except for attachments).

Extracted values

Envelope should contain

  • DTSTART (timestamp)
  • DTEND (timestamp)
  • RECURRENCENED( timestamp) - calculated in serializer
  • HASALARM (bool) => for korgac
  • SUBJECT (string)

=> Everything that is required to display the event in the calendar view.


KDE® and the K Desktop Environment® logo are registered trademarks of KDE e.V.Legal