View source for User:Markg85/KDE-Timemachine

{{Warning|This is a braindump! I'm just writing down what i think, how it should work and what would probably be needed to get this in place. There is '''no''' plan (yet) to implement this nor is this final by any means! Read it for your own pleasure or inspiration.}}

=Goal=
The goal for this is simple. There should be the possibility for our beloved users to use features like Apple's timemachine. This menas that applications can store "revisions". For example, if you make an edit in KEdit, close it, Make another edit then you have a revision where you could potentially move back to. Another example is Dolphin. Every change in Donplin should be a change in time and stored as such. So there you get the possibility to move back in time to see where you deleted your awesome photo. Yet another example, the calendar. You accidentally removed an even, but don't know where it was. You could simply use the "timemachine" to go back in time and see how your calendar looked like some time ago.
 
That is the intention with this "KDE-Timemachine" page. I hope this page will get updated over time to reflect new/other visions and perhaps make it a GSoC project in 2013.

=Storing revisions=
Revisions obviously need to be stored somewhere, but how? We have, at the very least, two different types of revisions that we need to store.

==File edits aka deltas==
In the event of text file editing the storing is quite simple. You have an original file, you open it, make edits and save it. Those edits become the delta of the currently saved file. You simply store that delta somewhere.

==Object edits==
Lets for the moment simply call everything that's not a text file an "object". Here is where it gets interesting. Imagine Dolphin. How would you store a change in the directory? You can't simply make a diff between 2 folders and store it's delta like you can with text files. You need to know what happened and store that. Here is where Zeitgeist "could" potentially come in. Zeitgeist stores events. If you rename a file, delete it, copy it or whatever, zeitgeist can know about it depending on the support. That is one part of the picture. Based on that one could compose a diff with "old name" -> "new name" and a timestamp. That is still not enough. You still need to record the actual change. So if for example an image got overwritten then the old image needs to be stored somewhere in order for timetravel stuff to go back and show you what you've deleted.
 
Another tricky issue is anything that's not text file based or dolphin (filesystem) based. Lets take the calendar example again. If you delete a calendar event then that deleted event should be stored in the timetravel stuff, but it's not a file. It are just parameters in some class and likely stored in Akonadi. When you delete such an event it is really deleted from that database. There is no way to restore that back (it's just an entry in a MySQL database iirc), so how can we ever store such things for timetravel purposes? This one is where the real challenge is.
 
Perhaps we need some interface for KDEPIM with an implementation for Akonadu and an implementation for "timetravel"..? More about this is below in Say hello to "LibTimetravel".

=Where do we store the data?=
Well, we already have quite a few storage mechanisms in KDE land. 
 
There is Nepomuk, used for storeing metadata and certainly not usable for storing file diffs or entire files.
 
There is Akonadi, this is used to store KDEPIM data, but this is in a MySQL database and again not suited for storing files or diffs.

There is Zeitgeist, that one is - to put it simple - a very advanced logger. Yet again not suitable for storing diffs or files.
 
None of those storage mechanisms are suited to have time machine capabilities. So storing diff data including files. Akonadi could - potentially - be changed since it uses MySQL to maintain a history dataset as well, but then you don't have that data in any other place then in a deeply hidden database somewhere. No, i think we need - sadly - another storage engine that is particularly suited for storing files and diffs. 
 
With this one i would like to follow by example. Thus apple. The data should be stored on a user pointed place. This could be anywhere! Somewhere on the network, an attached drive, an usb stick. You name it. The user should be able to simply say where he wants to have the data stored.

=How do we store the data?=
This could be done in 2 ways. Compressed or uncompressed. I personally would like to store this data as small as possible thus compressed. All the tools for decompressing and reading the data should be provided for multiple platforms and "just work". In the case of images i would store a binary diff of the image (the delta) and the last known image in full format. So the storage is in fact a kind of "backup" (though this kind of backup should not be used at "the" backup) as well.
 
The same is true for all other data in the timemachine. The delta's should be stored as compressed as possible with the latest version in full format. (full format is just the latest "version" of the file).
 
To be clear, the data that should be stored is the full revision set of some file in highly compressed state and the latest version of the same file in full format uncompressed just as it sits on the hard drive.

As for everything that's KDEPIM based. I don't know how we can store that externally.. It's tricky.

=Hello Kioslave!=
All file based operations (creating, deleting, editing ....) are already going through KIO like it should. I would not recommend changing the default KIO slaves to suddenly start making revisions of everything it handles. That will certainly get messy and perhaps even unstable for a while. What we should have is a whole new Kioslave that is handling timetravel. So this new kioslave should first:
Send the event (copy, edit, delete, whatever..) to the timetravel library (yep, more on that one just below here)
Once the timetravel part is done, send the data to the actual kioslave like it's hapening right now when you delete a file.
 
That way application developers can choose if they want to support timetraveling simply by using the kio timetravel slave.

=Say hello to "LibTimetravel"=
So there we have it, we need a new library. This library should contain all the logic for making diffs, storing them and everything else i wrote above this. The kioslave should use this library. There is not a lot more to say about this one.

=Implementing KDE-Timetravel=
The following is how i would imagine this to be implemented.
==Step 1==
As the very first step the kioslave has to be created. While making that the LibTimetravel also has to be created. And, obviously, an application to test this out (or an edited dolphin that uses the new kioslave). At this point is can be a dump library, simply storing files in some other place with a timestamp in it or something. Nothing very fancy is needed here, just the basics.
==Step 2==
Once the kioslave is done the next thing is to implement diff files between revisions. Binary diff files between binaries. Once this works, the KDE-Timetravel implementation is really getting of the ground and in a workable situation. This step should end with the ability to go back in time with dolphin.
==Step 3==
Next thing to do is restoring a file. Once this step is done you should be able to delete files in Dolphin, use KDE-Timetravel, go back some time and restore the deleted files. (and changed ones)
==Step 4==
At this point the basic functionality for KDE-Timetravel is there and is usable. Though it's still a bit fat in disc space usage. The thing to do here is invest which option has the best compression while maintaining speed. The diff and binary diff files should be compressed. Once this step is done the result should still be the same as in Step 3 only with a smaller disc space footprint.
==Step 5==
Now the tools have to be written to use the data on the timemachine anywhere. This one is going to be a very long lasting one since you have to write the tools for all kinds of data that is supported in a platform agnostic way. No dependencies on KDE for this one. Qt and perhaps some other libs are oke.
==Step 6==
At this point we're probably working on this stuff for about a year or so. Yet we still miss every single possibility to have a timemachine for KDEPIM data. Sadly i don't quite know how to resolve this at this moment, but you perhaps might know. Once this step is done, KDEPIM data should be stored on the timemachine as well. Good luck with it!
==Step 7==
Don't forget to also write the tools to see the KDEPIM data on the timemachine :)

=Done=
Here we are done making this timemachine and KDE has Timemachine support with a nice graphical tool to configure it and very easy for the user to use. The user can access all it's data everywhere with the provided tools and is very happy about it. It crashes never and certainly has no data loss issues.
 
How's that for a KDE GSoC idea :)