Projects/Digikam/GSOC2011/ZhangJieProposal

Proposal for digiKam--Clone Tool for Image Editor

Information about me

I'm Zhang Jie. As a student in Zhejiang University (ZJU), my research interests are in graphical image processing. I'm very interested in the Clone Tool for Image Editor by digiKam at GSoC 2011.This is the second year of my Master study period. I'd like to join in the open source.

I uses digiKam to organize my pictures. So I'm very familiar with this software. I have some experience that might be relevant to the project. I have learned QT for 2 years and I did a simple Image Editing software last year. There are many tools in the software, such as brush, eraser, scissors tool etc. To do the work I studied all the demoes in QT. Also, as a Lab project, I have written a plug-in for photoshop--generat mapping graph according to a picture. I use openCV library in the program. For programming language, I use C++.

I will spend more than 40 hours every week to work on the project. For the mentor to track my work , I will submit a progress record once a week. Also, we can have meetings on IRC. In fact, I submitted a progress record to my instructor every week when I worked on a software. As long as I can coordinate with the mentor, it really doesn't matter where he/she lives.

Here is my contact information:

IRC nick: zhjie

email address: [email protected]

postal address:China ,Hang Zhou, Zhejiang University, 38 Zheda Road, Fourth quarters

telephone number:+86 15268566953

Bye the way, my native language is Chinese, I live in Hang Zhou. As for time, we use Beijing Time Zone , that's 8 hours earlier than UTC.

Problem description

As a free, open-source, efficient image organizer, digiKam supports all major image file formats, and can organize collections of photographs in directory-based albums, or dynamic albums by date, timeline, or by tags. With the help of digiKam, people can organize their pictures in perfect order.What if you are not satisfy your pictures taken by camera? What if there is annoying dusts, spots, and other unwanted artefact in your picture? The thing you just need is a clone tool. You can clone from the same picture or other pictures a similar regional to patch the dusts etc.

Recent research has made it possible to seamless clone, clone the clear rectangle that you choose to the dust area, it can coordinate with the surrounding environment in the picture, seamlessly.

The aim of this project is to provide a clone tool like this to support picture editing.

Implementation plan

The project consists of two components:

A friendly interaction interface for clone tool

For convenient operation of the clone tool, I would like to design the tool as a brush. You can change the size of the brush as well as shape maybe. First chose a central region(without dusts, spots, and other unwanted artefact ), then draw on the dusts area in the picture use the brush. I want to make to tool just like Healing brush tool in Photoshop.

Clone

The basic method of clone is to remove defects in images . Copying directly is not advisable, as it cannot suit the environment in the image. The source rectangle may be lighter or has a different color comparing with the target area. What the project should achieve is to change the color and light of the source rectangle according to the target area. That is seamless clone.

Related Work in this area are as follows

In paper "Poisson Image Editing" , the authors achieve seamless cloning by solving a Poisson equation to fill in the selected area. The right hand side ’source’ term in the equation is Laplacian of the sampled image. Dirichlet boundary conditions make the solution continuous at the boundary. Source code of the paper is available.

Here are some results of "Poisson Image Editing": (To avoid a too big e-mail, I uploaded the result images to my flickr)

results from the paper [1] [2]

results got by the source code

source picture:[3]

maks picture:[4]

target picture:[5]

result picture:[6]

Todor Georgiev, in his paper "Photoshop Healing Brush:a Tool for Seamless Cloning" , proposed that continuity at the boundary is not always sufficient. He said if we want higher quality, seamless fill in requires continuity of derivatives. He use a fourth order PDE(partial differential equation) for solving that. That is just the method used Photoshop as Healing Brush.

Another paper "Poisson Matting", gives a approach called Poisson matting. As the matte is directly reconstructed from a continuous matte gradient field by solving Poisson equations using boundary information from a user-supplied trimap, the modified local result is seamlessly integrated into the final result. By interactively , people can manipulating the matte gradient field using a number of filtering tools until he or she is satisfied. This approach gives very good result especially in details such as hair. [7]

In "Coordinates for Instant Image Cloning", using the general framework of mean-value coordinates, the authors presented a new approach for seamless cloning of images and video, stitching, and matting. [8]

Deliverables

A clone tool to be able to remove quickly dusts, spots, and other unwanted artefact from an image.

Source code and the documentation.

Timeline

More than 40 hours is spent on the project every week. More details for timeline are as follows:

This week-by-week timeline provides a rough guideline of how the project will be done.

3 -- 16 May

Discuss with the mentor and make a determine on the design of the interaction interface. Make a deeper understanding with related documents and papers.

17 -- 30 May

Test and document existing code more thoroughly. Familiarize with the code and do some optimization.

31 May -- 6 June

Familiar with digiKam source code and get the framework design of the process started. At the same time, start with program documents writing.

7 -- 20 June

Determine algorithms used to do the seamless clone. Use QT to write the interface.

21 -- 27 June

Finish the functions that is respond to user actions.

28 June -- 11 July

Test the logical of the respond to user actions, make it more reasonable and more friendly.

12 -- 25 July

Finish the coding and program documents.

26 July -- 1 August

Test the hole project and the program documentation more thoroughly.

2 -- 16 August

Further refine tests and documentation for the whole project.