Development/Tutorials/Metadata/Nepomuk/TipsAndTricks - Revision history

AnneW: Marked this version for translation

2012-12-14T10:30:14Z

Marked this version for translation

AnneW: Prepare for translation

2012-12-14T10:29:59Z

Prepare for translation

Vishesh: Removed cruft

2012-08-25T21:54:34Z

Removed cruft

Vishesh at 15:58, 23 August 2012

2012-08-23T15:58:37Z

Vishesh: /* Debugging the created data */ Removed information about sopranocmd - The Dbus model no longer exists, and you cannot monitor for statements any more.

2012-08-08T17:11:33Z

‎Debugging the created data: Removed information about sopranocmd - The Dbus model no longer exists, and you cannot monitor for statements any more.

Vishesh: Remove stupid stuff

2012-08-01T22:35:06Z

Remove stupid stuff

Vishesh at 10:23, 30 July 2012

2012-07-30T10:23:13Z

AnneW at 13:12, 18 July 2012

2012-07-18T13:12:08Z

Lvsouza: /* Using ontology URIs in your code */

2012-03-07T14:53:47Z

‎Using ontology URIs in your code

Rabauke: virtuoso-t debugging.

2012-01-30T17:57:53Z

virtuoso-t debugging.

@@ Line 2: / Line 2: @@
 <translate>
-== Using ontology URIs in your code  ==
+== Using ontology URIs in your code  == <!--T:1-->
 One often needs the URI of a specific class or a specific property in ones code. And not all ontologies are provided by the very convenient [http://soprano.sourceforge.net/apidox/stable/namespaceSoprano_1_1Vocabulary.html Soprano::Vocabulary] namespace.
 The solution is rather simple: create your own vocabulary namespaces by using Soprano's own onto2vocabularyclass command line tool. It can generate convenient vocabulary namespaces for you. The Soprano documentation shows how to [http://soprano.sourceforge.net/apidox/trunk/soprano_devel_tools.html use it manually] or even simpler with a [http://soprano.sourceforge.net/apidox/trunk/soprano_howto.html#cmake_magic simple CMake macro].
 <br />
-== Mind the Difference between QString and QUrl  ==
+== Mind the Difference between QString and QUrl  == <!--T:5-->
 [http://api.kde.org/4.x-api/kdelibs-apidocs/nepomuk/html/classNepomuk_1_1Resource.html Nepomuk::Resource] provides two constructors: one taking a {{qt|QString}} as identifier or URI and one taking a {{qt|QUrl}}.
 The latter one is really simple: the given URI is used as the resource URI. If the resource exists, its data is used, otherwise it will be created with exactly that URI.
 The {{qt|QString}} one is a bit trickier. It will try to be clever about the parameter and see if it is a URI. If no resource with that URI (if it is a URI) exists, it is interpreted as an identifier ([http://www.semanticdesktop.org/ontologies/nao/#mozTocId802441 nao:identifier]). Resource checks if a resource with that identifier exists. If so, its data is loaded, if not, a new resource with a random URI and that string as identifier is created.
 However, '''be aware that nothing is written to '''Nepomuk''' until the first writing call to Resource such as setProperty or addType.'''
 <br />
-== Debugging the created data  ==
+== Debugging the created data  == <!--T:11-->
 Soprano provides a command line client to connect to the storage service. It's called <tt>sopranocmd</tt>. It provides all the features one needs to debug data. It is recommended that you only use sopranocmd for running queries.
 Running sopranocmd is cumbersome because of the large number of arguments it requires. This can be made simpler by adding the following alias -
 <syntaxhighlight lang="bash">alias nepomukcmd="sopranocmd --socket `kde4-config --path socket`nepomuk-socket --model main --nrl"</syntaxhighlight>
 For example -
 <syntaxhighlight lang="text">
@@ Line 37: / Line 48: @@
 </syntaxhighlight>
-=== Using Konqueror  ===
+=== Using Konqueror  === <!--T:15-->
 In the [http://websvn.kde.org:80/trunk/playground/base/nepomuk-kde/kioslaves/nepomuk/ Nepomuk playground] repository lives a KIO slave which can handle the ''nepomuk:/'' protocol. It will display all properties of a Nepomuk resource including its links to other resources and the backlinks. This is a convenient way of looking at the Nepomuk data. The KIO slave even support removal of resources.
 [[Image:Nepomuk kio slave.png|560px]]
 <br />
-=== Using NepomukShell  ===
+=== Using NepomukShell  === <!--T:19-->
 NepomukShell is a maintenance and debugging tool, which lives in its own git repository at  [https://projects.kde.org/projects/extragear/utils/nepomukshell nepomukshell]. It is a simple tool that let's one browse all resources in '''Nepomuk'''. Additionally it allows to create subclasses and properties and remove resources.
 {{Remember|2='''Caution'''|1=Do only create subclasses and properties from PIMO classes and properties!}}
 [[Image:Pimoshell.png|560px]]
-== Constructing SPARQL queries  ==
+== Constructing SPARQL queries  == <!--T:22-->
 {{Tip|1= In most cases the [http://api.kde.org/4.x-api/kdelibs-apidocs/nepomuk/html/namespaceNepomuk_1_1Query.html Nepomuk Query API] should be enough and prevent you from writing your own SPARQL which is hard to debug.}}
 Whenever doing something a bit fancier with '''Nepomuk''' one has to use SPARQL queries via <syntaxhighlight lang="cpp-qt">
 Nepomuk::ResourceManager::instance()->mainModel()
@@ Line 62: / Line 80: @@
 </syntaxhighlight>
 Constructing these queries can be a bit cumbersome since one has to use a lot of class and property URIs from different ontologies. Also literals have to be formatted according to the N3 syntax used in SPARQL. Luckily Soprano provides the necessary tools to do exactly that: <br />[http://soprano.sourceforge.net/apidox/trunk/classSoprano_1_1Node.html#ad4c8ab988ae7d9fd587027087b593e4 Soprano::Node::toN3], <br />[http://soprano.sourceforge.net/apidox/trunk/classSoprano_1_1Node.html#d1c2618a28a13c6eac042ddccbf78e6a Soprano::Node::resourceToN3], and <br />[http://soprano.sourceforge.net/apidox/trunk/classSoprano_1_1Node.html#a66acf156e82b866114d90cd0c9ce13c Soprano::Node::literalToN3] take care of all formatting and percent-encoding you need. Using those methods the code to create queries might look ugly but the resulting queries are more likely to be correctly encoded and introduce less code duplication.
 Typically one would use QString::arg like so (be aware that the standard prefixes are NOT supported out-of-the-box as with sopranocmd):
 <syntaxhighlight lang="cpp-qt">
 using namespace Soprano;
 QString myQuery
       = QString("select ?r where { "
@@ Line 77: / Line 99: @@
         .arg(Node::literalToN3("foobar")));
 </syntaxhighlight>
 This will create the same query we used above only using no hard-coded components whatsoever.
-== Debugging virtuoso-t  ==
+== Debugging virtuoso-t  == <!--T:31-->
 If virtuoso-t consumes a lot of CPU resources but there are no active queries analysis has to go a bit deeper. Virtuoso is started through Soprano with certain parameters which are set in a temporary ini-file (/tmp/virtuoso_XXXX.ini). Soprano needs to be modified manually to start Virtuoso with different parameters in the ini-file, e.g. to improve virtuoso-t's behaviour by modifying backends/virtuoso/virtuosocontroller.cpp (Soprano) and setting NumberOfBuffers to 40000 (line 344) and SchedulerInterval to 0 (line 350).
 After re-compiling soprano one has to attach gdb to virtuoso-t as soon as it starts consuming CPU and create a full threaded backtrace:
 <syntaxhighlight lang="cpp-qt">
 set logging file /tmp/virtuoso-t.out
@@ Line 93: / Line 120: @@
 </syntaxhighlight>
 Note: The above settings should only be used for debugging!
 [[Category:Documentation]]
 [[Category:Development]]
 [[Category:Tutorials]]
 </translate>

@@ Line 1: / Line 1: @@
+<languages />
 == Using ontology URIs in your code  ==
@@ Line 7: / Line 8: @@
 The solution is rather simple: create your own vocabulary namespaces by using Soprano's own onto2vocabularyclass command line tool. It can generate convenient vocabulary namespaces for you. The Soprano documentation shows how to [http://soprano.sourceforge.net/apidox/trunk/soprano_devel_tools.html use it manually] or even simpler with a [http://soprano.sourceforge.net/apidox/trunk/soprano_howto.html#cmake_magic simple CMake macro].
-<br>
+<br />
 == Mind the Difference between QString and QUrl  ==
@@ Line 17: / Line 18: @@
 The {{qt|QString}} one is a bit trickier. It will try to be clever about the parameter and see if it is a URI. If no resource with that URI (if it is a URI) exists, it is interpreted as an identifier ([http://www.semanticdesktop.org/ontologies/nao/#mozTocId802441 nao:identifier]). Resource checks if a resource with that identifier exists. If so, its data is loaded, if not, a new resource with a random URI and that string as identifier is created.
-However, '''be aware that nothing is written to Nepomuk until the first writing call to Resource such as setProperty or addType.'''
+However, '''be aware that nothing is written to '''Nepomuk''' until the first writing call to Resource such as setProperty or addType.'''
-<br>
+<br />
 == Debugging the created data  ==
-Soprano provides a command line client to connect to the storage service. It's called <code>sopranocmd</code>. It provides all the features one needs to debug data. It is recommended that you only use sopranocmd for running queries.
+Soprano provides a command line client to connect to the storage service. It's called <tt>sopranocmd</tt>. It provides all the features one needs to debug data. It is recommended that you only use sopranocmd for running queries.
 Running sopranocmd is cumbersome because of the large number of arguments it requires. This can be made simpler by adding the following alias -
@@ Line 42: / Line 43: @@
 [[Image:Nepomuk kio slave.png|560px]]
-<br>
+<br />
 === Using NepomukShell  ===
-NepomukShell is a maintenance and debugging tool, which lives in its own git repository at  [https://projects.kde.org/projects/extragear/utils/nepomukshell nepomukshell]. It is a simple tool that let's one browse all resources in Nepomuk. Additionally it allows to create subclasses and properties ('''Caution: do only create subclasses and properties from PIMO classes and properties!''') and remove resources.
+NepomukShell is a maintenance and debugging tool, which lives in its own git repository at  [https://projects.kde.org/projects/extragear/utils/nepomukshell nepomukshell]. It is a simple tool that let's one browse all resources in '''Nepomuk'''. Additionally it allows to create subclasses and properties and remove resources.
 [[Image:Pimoshell.png|560px]]
@@ Line 52: / Line 54: @@
 == Constructing SPARQL queries  ==
-'''Hint: In most cases the [http://api.kde.org/4.x-api/kdelibs-apidocs/nepomuk/html/namespaceNepomuk_1_1Query.html Nepomuk Query API] should be enough and prevent you from writing your own SPARQL which is hard to debug.'''
+{{Tip|1= In most cases the [http://api.kde.org/4.x-api/kdelibs-apidocs/nepomuk/html/namespaceNepomuk_1_1Query.html Nepomuk Query API] should be enough and prevent you from writing your own SPARQL which is hard to debug.}}
-Whenever doing something a bit fancier with Nepomuk one has to use SPARQL queries via <syntaxhighlight lang="cpp-qt">
+Whenever doing something a bit fancier with '''Nepomuk''' one has to use SPARQL queries via <syntaxhighlight lang="cpp-qt">
 Nepomuk::ResourceManager::instance()->mainModel()
      ->executeQuery( myQueryString,
                      Soprano::Query::QueryLanguageSparql );
-</syntaxhighlight> Constructing these queries can be a bit cumbersome since one has to use a lot of class and property URIs from different ontologies. Also literals have to be formatted according to the N3 syntax used in SPARQL. Luckily Soprano provides the necessary tools to do exactly that: [http://soprano.sourceforge.net/apidox/trunk/classSoprano_1_1Node.html#ad4c8ab988ae7d9fd587027087b593e4 Soprano::Node::toN3], [http://soprano.sourceforge.net/apidox/trunk/classSoprano_1_1Node.html#d1c2618a28a13c6eac042ddccbf78e6a Soprano::Node::resourceToN3], and [http://soprano.sourceforge.net/apidox/trunk/classSoprano_1_1Node.html#a66acf156e82b866114d90cd0c9ce13c Soprano::Node::literalToN3] take care of all formatting and percent-encoding you need. Using those methods the code to create queries might look ugly but the resulting queries are more likely to be correctly encoded and introduce less code duplication.
+</syntaxhighlight>
 Typically one would use QString::arg like so (be aware that the standard prefixes are NOT supported out-of-the-box as with sopranocmd):
@@ Line 90: / Line 94: @@
 Note: The above settings should only be used for debugging!

@@ Line 1: / Line 1: @@
 == Using ontology URIs in your code  ==
@@ Line 84: / Line 76: @@
 This will create the same query we used above only using no hard-coded components whatsoever.
 == Debugging virtuoso-t  ==

@@ Line 8: / Line 8: @@
 [[../RDFIntroduction|RDF and Ontologies in Nepomuk]]
 }}
 == Using ontology URIs in your code  ==

@@ Line 41: / Line 41: @@
 == Debugging the created data  ==
-=== Using sopranocmd  ===
+Soprano provides a command line client to connect to the storage service. It's called <code>sopranocmd</code>. It provides all the features one needs to debug data. It is recommended that you only use sopranocmd for running queries.
-When using Nepomuk one creates a lot of RDF statements in the Nepomuk RDF storage. It is often of interest to check which data has been created, if statements have been correctly created or simply look at existing data.
+Running sopranocmd is cumbersome because of the large number of arguments it requires. This can be made simpler by adding the following alias -
 <syntaxhighlight lang="text">
-# sopranocmd --dbus org.kde.NepomukStorage --model main <command> \
+# nepomukcmd query \
-    <parameters>
-</syntaxhighlight>
-−
-If one wanted to list all the resources that have been tagged with the tag whose resource URI is nepomuk:/foobar one would use the following command:
-−
-<syntaxhighlight lang="text">
-# sopranocmd --dbus org.kde.NepomukStorage --model main list \
-    "" "" "<nepomuk:/foobar>"
-</syntaxhighlight>
-−
-or one would use a SPARQL query ('''sopranocmd supports the standard URI prefixes out of the box'''):
-−
-<syntaxhighlight lang="text">
-# sopranocmd --dbus org.kde.NepomukStorage --model main query \
      "select ?r where { ?r nao:hasTag ?tag . \
                         ?tag nao:prefLabel 'foobar'^^xsd:string . }"
 </syntaxhighlight>
 === Using Konqueror  ===

@@ Line 79: / Line 79: @@
 As a shortcut add the following to your .bashrc to avoid having to type in the dbus and model parameters all the time:
 <syntaxhighlight lang="text">alias nepomukcmd="sopranocmd --socket `kde4-config --path socket`nepomuk-socket --model main --nrl"</syntaxhighlight>
@@ Line 156: / Line 155: @@
 == Listening to changes in the database  ==
-Nepomuk's database does only contain statements, i.e. quadruples. To date Soprano's Model does provide four signals which can be used to monitor new and removed statements: statementsAdded and statementsRemoved as well as their counterparts which have as paramter the added or removed statement.
+Write about the Resource Watcher.
-−
-It is recommended to use Soprano's [http://soprano.sourceforge.net/apidox/stable/classSoprano_1_1Util_1_1SignalCacheModel.html SignalCacheModel] when listening to changes to prevent a slowdown of the whole system as the signals are emitted for each statement:
-−
-<syntaxhighlight lang="cpp-qt">
-Soprano::Util::SignalCacheModel* scm = new Soprano::Util::SignalCacheModel(
-    Nepomuk::ResourceManager::instance()->mainModel() );
-−
-connect( scm, SIGNAL(statementsAdded()),
-         this, SLOT(slotStatementsAdded()) );
-connect( scm, SIGNAL(statementsRemoved()),
-         this, SLOT(slotStatementsRemoved()) );
-</syntaxhighlight>
 == Remove all Strigi-indexed data  ==
@@ Line 175: / Line 162: @@
 The little command below removes all data created by Strigi (caution: this could take a long time):
 <syntaxhighlight lang="text">
@@ Line 181: / Line 170: @@
    do nepomukcmd rmgraph "$a"; done
 </syntaxhighlight>

← Older revision		Revision as of 13:12, 18 July 2012
Line 1:		Line 1:
−	~~{{Template:I18n/Language Navigation Bar\|Development/Tutorials/Metadata/Nepomuk/TipsAndTricks}}~~	+

	{{TutorialBrowser\|		{{TutorialBrowser\|

← Older revision		Revision as of 17:57, 30 January 2012
Line 201:		Line 201:
	export PATH		export PATH
	</syntaxhighlight>		</syntaxhighlight>
		+
		+	<br>
		+
		+	== Debugging virtuoso-t ==
		+
		+	If virtuoso-t consumes a lot of CPU resources but there are no active queries analysis has to go a bit deeper. Virtuoso is started through Soprano with certain parameters which are set in a temporary ini-file (/tmp/virtuoso_XXXX.ini). Soprano needs to be modified manually to start Virtuoso with different parameters in the ini-file, e.g. to improve virtuoso-t's behaviour by modifying backends/virtuoso/virtuosocontroller.cpp (Soprano) and setting NumberOfBuffers to 40000 (line 344) and SchedulerInterval to 0 (line 350).
		+
		+	After re-compiling soprano one has to attach gdb to virtuoso-t as soon as it starts consuming CPU and create a full threaded backtrace:
		+
		+	<syntaxhighlight lang="cpp-qt">
		+	set logging file /tmp/virtuoso-t.out
		+	set logging on
		+	thread apply all bt full
		+	</syntaxhighlight>
		+
		+	Note: The above settings should only be used for debugging!