Projects/Nepomuk/IndexingPlugin - Revision history

Vishesh at 18:08, 23 December 2012

2012-12-23T18:08:42Z

AnneW: Marked this version for translation

2012-12-12T13:28:35Z

Marked this version for translation

AnneW: Prepare for translation

2012-12-12T13:28:21Z

Prepare for translation

Vishesh: /* Testing the Indexer */

2012-11-07T17:09:37Z

‎Testing the Indexer

Vishesh: /* Extractor Plugin */

2012-11-05T16:49:38Z

‎Extractor Plugin

Vishesh: /* Saving the Extracted Data */

2012-11-05T16:48:52Z

‎Saving the Extracted Data

Vishesh: /* Testing the Indexer */

2012-11-05T16:48:30Z

‎Testing the Indexer

Vishesh: /* Required Files */

2012-11-05T16:44:06Z

‎Required Files

Vishesh: Created page with "File Indexing has gone through a major overhaul in 4.10. We no longer rely on strigi. This means that we need to write our own file indexer from scratch. However writing a fil..."

2012-11-05T16:30:15Z

Created page with "File Indexing has gone through a major overhaul in 4.10. We no longer rely on strigi. This means that we need to write our own file indexer from scratch. However writing a fil..."

New page

File Indexing has gone through a major overhaul in 4.10. We no longer rely on strigi. This means that we need to write our own file indexer from scratch. However writing a file indexer is very simple.

Currently, there is no public interface for the indexing plugins. There might be one for 4.10, but we aren't sure right now.

= Extractor Plugin =
In order to write a file indexer, we have to write a plugin derived from <tt>Nepomuk2::ExtractorPlugin</tt>. We are required to implement two simple functions -

<syntaxhighlight lang="cpp-qt">
class NEPOMUK_EXPORT ExtractorPlugin : public QObject
{
Q_OBJECT
public:
ExtractorPlugin(QObject* parent);
virtual ~ExtractorPlugin();

virtual QStringList mimetypes() = 0;
virtual SimpleResourceGraph extract(const QUrl& resUri, const QUrl& fileUrl, const QString& mimeType) = 0;
};
</syntaxhighlight>

These two functions are <tt>mimetypes</tt> and <tt>extract</tt>. Each plugin can act on a certain set of mimetypes. Each plugin simply needs to list out all the mimetypes they support.

The second function <tt>extract</tt> is the heart of the extractor. You are provided with the mimetype and the url of the file. The file can be read and information can be extracted from it.

== Saving the Extracted Data ==

The Nepomuk Extractors are based around two simple classes <tt>SimpleResource</tt> and <tt>SimpleResourceGraph</tt>. The SimpleResourceGraph is just a collection of <tt>SimpleResource</tt>s. A <tt>SimpleResource</tt> is just a collection of (key, value) pairs which contain the properties of that particular resource.

The main file resource has a resource uri which is passed as a parameter. It can be used as follows -

<syntaxhighlight lang="cpp-qt">
SimpleResource fileRes( resUri );
fileRes.addType( NFO::PlainTextDocument() );
fileRes.addProperty( NIE::plainTextContent(), contents );
fileRes.addProperty( NFO::wordCount(), words );
fileRes.addProperty( NFO::lineCount(), lines );
fileRes.addProperty( NFO::characterCount(), characters );
</syntaxhighlight>

This <tt>fileRes</tt> can then be added to a <tt>SimpleResourceGraph</tt> and returned. It will then be saved in Nepomuk.

Another simple example of a music file -

<syntaxhighlight lang="cpp-qt">
....
</syntaxhighlight>

= Required Files =

= Testing the Indexer =

@@ Line 64: / Line 64: @@
 <!--T:17-->
-The Indexer is generally automatically called when it detects new files should be indexed. It however can also be forcibly called by running <tt>nepomukindexer --debug fileUrl</tt> on a file.
+The Indexer is generally automatically called when it detects new files should be indexed. It however can also be forcibly called by running <tt>nepomukindexer fileUrl</tt> on a file.
-−
-<!--T:18-->
-The extra <tt>--debug</tt> is required because normally, the nepomukindexer process only adds details from the plugins. The basic information about the file - It's url, mimetype, etc, and supposed to already exist. The debug option adds that information as well.
 === Viewing the indexed information === <!--T:19-->

@@ Line 2: / Line 2: @@
 <translate>
-== Status of Indexing ==
+== Status of Indexing == <!--T:1-->
 File Indexing has gone through a major overhaul in 4.10. We no longer rely on '''strigi'''. This means that we need to write our own file indexer from scratch. However writing a file indexer is very simple.
 Currently, there is no public interface for the indexing plugins. There might be one for 4.10, but we aren't sure right now.
-== Extractor Plugin ==
+== Extractor Plugin == <!--T:4-->
 In order to write a file indexer, we have to write a plugin derived from <tt>Nepomuk2::ExtractorPlugin</tt>.  We are required to implement two simple functions -
 <syntaxhighlight lang="cpp-qt">
 class NEPOMUK_EXPORT ExtractorPlugin : public QObject
@@ Line 19: / Line 22: @@
      virtual ~ExtractorPlugin();
-     virtual QStringList mimetypes() = 0;
+     <!--T:6-->
      virtual SimpleResourceGraph extract(const QUrl& resUri, const QUrl& fileUrl, const QString& mimeType) = 0;
 };
 </syntaxhighlight>
 These two functions are <tt>mimetypes</tt> and <tt>extract</tt>. Each plugin can act on a certain set of mimetypes. Each plugin simply needs to list out all the mimetypes they support.
 The second function <tt>extract</tt> is the heart of the extractor. You are provided with the mimetype and the url of the file. The file can be read and information can be extracted from it.
-=== Saving the Extracted Data ===
+=== Saving the Extracted Data === <!--T:9-->
 The Nepomuk Extractors are based around two simple classes <tt>SimpleResource</tt> and <tt>SimpleResourceGraph</tt>. The SimpleResourceGraph is just a collection of <tt>SimpleResource</tt>s. A <tt>SimpleResource</tt> is just a collection of (key, value) pairs which contain the properties of that particular resource.
 The main file resource has a resource uri which is passed as a parameter. It can be used as follows -
 <syntaxhighlight lang="cpp-qt">
      SimpleResource fileRes( resUri );
@@ Line 44: / Line 53: @@
 This <tt>fileRes</tt> can then be added to a <tt>SimpleResourceGraph</tt> and returned. It will then be saved in Nepomuk.
-== Required Files ==
+== Required Files == <!--T:14-->
 Since the plugin interface still isn't public. It would be best to directly contribute to nepomuk-core. The relevant code can be found at nepomuk-core/services/fileindexer/indexer/.
-== Testing the Indexer ==
+== Testing the Indexer == <!--T:16-->
 The Indexer is generally automatically called when it detects new files should be indexed. It however can also be forcibly called by running <tt>nepomukindexer --debug fileUrl</tt> on a file.
 The extra <tt>--debug</tt> is required because normally, the nepomukindexer process only adds details from the plugins. The basic information about the file - It's url, mimetype, etc, and supposed to already exist. The debug option adds that information as well.
-=== Viewing the indexed information ===
+=== Viewing the indexed information === <!--T:19-->
 The most commonly used method is the sidebar in Dolphin. However, one can also use this [[Projects/Nepomuk/NepomukShow| nifty tool]] to view the data.
-=== Errors ===
+=== Errors === <!--T:21-->
 It might be common to get errors that a properties range/domain/cardinality is not being followed. These errors occur when the ontologies are not being properly followed. In that case it would be best to look where you're adding that property and if it actually has the correct domain/range/cardinality.
 The ontologies can be found over here - http://oscaf.sourceforge.net/
 [[Category:Documentation]]
 [[Category:Development]]
 [[Category:Tutorials]]
 </translate>

@@ Line 1: / Line 1: @@
-File Indexing has gone through a major overhaul in 4.10. We no longer rely on strigi. This means that we need to write our own file indexer from scratch. However writing a file indexer is very simple.
+<languages />
 Currently, there is no public interface for the indexing plugins. There might be one for 4.10, but we aren't sure right now.
-= Extractor Plugin =
+== Extractor Plugin ==
 In order to write a file indexer, we have to write a plugin derived from <tt>Nepomuk2::ExtractorPlugin</tt>.  We are required to implement two simple functions -
@@ Line 23: / Line 28: @@
 The second function <tt>extract</tt> is the heart of the extractor. You are provided with the mimetype and the url of the file. The file can be read and information can be extracted from it.
-== Saving the Extracted Data ==
+=== Saving the Extracted Data ===
 The Nepomuk Extractors are based around two simple classes <tt>SimpleResource</tt> and <tt>SimpleResourceGraph</tt>. The SimpleResourceGraph is just a collection of <tt>SimpleResource</tt>s. A <tt>SimpleResource</tt> is just a collection of (key, value) pairs which contain the properties of that particular resource.
@@ Line 41: / Line 46: @@
 This <tt>fileRes</tt> can then be added to a <tt>SimpleResourceGraph</tt> and returned. It will then be saved in Nepomuk.
-= Required Files =
+== Required Files ==
 Since the plugin interface still isn't public. It would be best to directly contribute to nepomuk-core. The relevant code can be found at nepomuk-core/services/fileindexer/indexer/.
-= Testing the Indexer =
+== Testing the Indexer ==
 The Indexer is generally automatically called when it detects new files should be indexed. It however can also be forcibly called by running <tt>nepomukindexer --debug fileUrl</tt> on a file.
@@ Line 51: / Line 56: @@
 The extra <tt>--debug</tt> is required because normally, the nepomukindexer process only adds details from the plugins. The basic information about the file - It's url, mimetype, etc, and supposed to already exist. The debug option adds that information as well.
-== Viewing the indexed information ==
+=== Viewing the indexed information ===
 The most commonly used method is the sidebar in Dolphin. However, one can also use this [[Projects/Nepomuk/NepomukShow| nifty tool]] to view the data.
-== Errors ==
+=== Errors ===
 It might be common to get errors that a properties range/domain/cardinality is not being followed. These errors occur when the ontologies are not being properly followed. In that case it would be best to look where you're adding that property and if it actually has the correct domain/range/cardinality.
 The ontologies can be found over here - http://oscaf.sourceforge.net/

← Older revision		Revision as of 17:09, 7 November 2012
Line 50:		Line 50:

	The extra <tt>--debug</tt> is required because normally, the nepomukindexer process only adds details from the plugins. The basic information about the file - It's url, mimetype, etc, and supposed to already exist. The debug option adds that information as well.		The extra <tt>--debug</tt> is required because normally, the nepomukindexer process only adds details from the plugins. The basic information about the file - It's url, mimetype, etc, and supposed to already exist. The debug option adds that information as well.
		+
		+	== Viewing the indexed information ==
		+
		+	The most commonly used method is the sidebar in Dolphin. However, one can also use this [[Projects/Nepomuk/NepomukShow\| nifty tool]] to view the data.

	== Errors ==		== Errors ==

← Older revision		Revision as of 16:48, 5 November 2012
Line 40:		Line 40:

	This <tt>fileRes</tt> can then be added to a <tt>SimpleResourceGraph</tt> and returned. It will then be saved in Nepomuk.		This <tt>fileRes</tt> can then be added to a <tt>SimpleResourceGraph</tt> and returned. It will then be saved in Nepomuk.
−
−	~~Another simple example of a music file -~~
−
−	~~<syntaxhighlight lang="cpp-qt">~~
−	~~....~~
−	~~</syntaxhighlight>~~

	= Required Files =		= Required Files =

← Older revision		Revision as of 16:48, 5 November 2012
Line 52:		Line 52:

	= Testing the Indexer =		= Testing the Indexer =
		+
		+	The Indexer is generally automatically called when it detects new files should be indexed. It however can also be forcibly called by running <tt>nepomukindexer --debug fileUrl</tt> on a file.
		+
		+	The extra <tt>--debug</tt> is required because normally, the nepomukindexer process only adds details from the plugins. The basic information about the file - It's url, mimetype, etc, and supposed to already exist. The debug option adds that information as well.
		+
		+	== Errors ==
		+
		+	It might be common to get errors that a properties range/domain/cardinality is not being followed. These errors occur when the ontologies are not being properly followed. In that case it would be best to look where you're adding that property and if it actually has the correct domain/range/cardinality.
		+
		+	The ontologies can be found over here - http://oscaf.sourceforge.net/

← Older revision		Revision as of 16:44, 5 November 2012
Line 48:		Line 48:

	= Required Files =		= Required Files =
		+
		+	Since the plugin interface still isn't public. It would be best to directly contribute to nepomuk-core. The relevant code can be found at nepomuk-core/services/fileindexer/indexer/.

	= Testing the Indexer =		= Testing the Indexer =