Difference between revisions of "Development/Tutorials/Metadata/Nepomuk/TipsAndTricks"

Jump to: navigation, search
m (Text replace - "<code>" to "<syntaxhighlight lang="text">")
m (Text replace - "</code>" to "</syntaxhighlight>")
Line 15: Line 15:
 
<code cppqt="cppqt">
 
<code cppqt="cppqt">
 
Nepomuk::ResourceManager::instance()->init();
 
Nepomuk::ResourceManager::instance()->init();
</code>  
+
</syntaxhighlight>  
  
 
<br>  
 
<br>  
Line 52: Line 52:
 
# sopranocmd --dbus org.kde.NepomukStorage --model main <command> \
 
# sopranocmd --dbus org.kde.NepomukStorage --model main <command> \
 
     <parameters>
 
     <parameters>
</code>  
+
</syntaxhighlight>  
  
 
If one wanted to list all the resources that have been tagged with the tag whose resource URI is nepomuk:/foobar one would use the following command:  
 
If one wanted to list all the resources that have been tagged with the tag whose resource URI is nepomuk:/foobar one would use the following command:  
Line 59: Line 59:
 
# sopranocmd --dbus org.kde.NepomukStorage --model main list \
 
# sopranocmd --dbus org.kde.NepomukStorage --model main list \
 
     "" "" "<nepomuk:/foobar>"
 
     "" "" "<nepomuk:/foobar>"
</code>  
+
</syntaxhighlight>  
  
 
or one would use a SPARQL query ('''sopranocmd supports the standard URI prefixes out of the box'''):  
 
or one would use a SPARQL query ('''sopranocmd supports the standard URI prefixes out of the box'''):  
Line 67: Line 67:
 
     "select ?r where { ?r nao:hasTag ?tag . \
 
     "select ?r where { ?r nao:hasTag ?tag . \
 
                       ?tag nao:prefLabel 'foobar'^^xsd:string . }"
 
                       ?tag nao:prefLabel 'foobar'^^xsd:string . }"
</code>  
+
</syntaxhighlight>  
  
 
To monitor all statements that are added and removed from the Nepomuk storage one would simply use the following command (as with ''list'' one can specify a filter to only list the added and removed statements one is interested in): <syntaxhighlight lang="text">
 
To monitor all statements that are added and removed from the Nepomuk storage one would simply use the following command (as with ''list'' one can specify a filter to only list the added and removed statements one is interested in): <syntaxhighlight lang="text">
 
# sopranocmd --dbus org.kde.NepomukStorage --model main monitor
 
# sopranocmd --dbus org.kde.NepomukStorage --model main monitor
</code>  
+
</syntaxhighlight>  
  
<syntaxhighlight lang="text"># sopranocmd --help</code> is your friend for all details.  
+
<syntaxhighlight lang="text"># sopranocmd --help</syntaxhighlight> is your friend for all details.  
  
 
==== nepomukcmd ====
 
==== nepomukcmd ====
Line 80: Line 80:
  
 
'''KDE 4.6 and beyond:'''
 
'''KDE 4.6 and beyond:'''
<syntaxhighlight lang="text">alias nepomukcmd="sopranocmd --socket `kde4-config --path socket`nepomuk-socket --model main --nrl"</code>  
+
<syntaxhighlight lang="text">alias nepomukcmd="sopranocmd --socket `kde4-config --path socket`nepomuk-socket --model main --nrl"</syntaxhighlight>  
  
 
'''before KDE 4.6:'''
 
'''before KDE 4.6:'''
<syntaxhighlight lang="text">alias nepomukcmd="sopranocmd --socket `kde4-config --localprefix`/share/apps/nepomuk/socket --model main --nrl"</code>  
+
<syntaxhighlight lang="text">alias nepomukcmd="sopranocmd --socket `kde4-config --localprefix`/share/apps/nepomuk/socket --model main --nrl"</syntaxhighlight>  
  
 
''(Be aware that the --nrl parameter is only available in Soprano 2.3.63 and above.)''
 
''(Be aware that the --nrl parameter is only available in Soprano 2.3.63 and above.)''
Line 109: Line 109:
 
     ->executeQuery( myQueryString,  
 
     ->executeQuery( myQueryString,  
 
                     Soprano::Query::QueryLanguageSparql );
 
                     Soprano::Query::QueryLanguageSparql );
</code> Constructing these queries can be a bit cumbersome since one has to use a lot of class and property URIs from different ontologies. Also literals have to be formatted according to the N3 syntax used in SPARQL. Luckily Soprano provides the necessary tools to do exactly that: [http://soprano.sourceforge.net/apidox/trunk/classSoprano_1_1Node.html#ad4c8ab988ae7d9fd587027087b593e4 Soprano::Node::toN3], [http://soprano.sourceforge.net/apidox/trunk/classSoprano_1_1Node.html#d1c2618a28a13c6eac042ddccbf78e6a Soprano::Node::resourceToN3], and [http://soprano.sourceforge.net/apidox/trunk/classSoprano_1_1Node.html#a66acf156e82b866114d90cd0c9ce13c Soprano::Node::literalToN3] take care of all formatting and percent-encoding you need. Using those methods the code to create queries might look ugly but the resulting queries are more likely to be correctly encoded and introduce less code duplication.  
+
</syntaxhighlight> Constructing these queries can be a bit cumbersome since one has to use a lot of class and property URIs from different ontologies. Also literals have to be formatted according to the N3 syntax used in SPARQL. Luckily Soprano provides the necessary tools to do exactly that: [http://soprano.sourceforge.net/apidox/trunk/classSoprano_1_1Node.html#ad4c8ab988ae7d9fd587027087b593e4 Soprano::Node::toN3], [http://soprano.sourceforge.net/apidox/trunk/classSoprano_1_1Node.html#d1c2618a28a13c6eac042ddccbf78e6a Soprano::Node::resourceToN3], and [http://soprano.sourceforge.net/apidox/trunk/classSoprano_1_1Node.html#a66acf156e82b866114d90cd0c9ce13c Soprano::Node::literalToN3] take care of all formatting and percent-encoding you need. Using those methods the code to create queries might look ugly but the resulting queries are more likely to be correctly encoded and introduce less code duplication.  
  
 
Typically one would use QString::arg like so (be aware that the standard prefixes are NOT supported out-of-the-box as with sopranocmd):  
 
Typically one would use QString::arg like so (be aware that the standard prefixes are NOT supported out-of-the-box as with sopranocmd):  
Line 124: Line 124:
 
       .arg(Node::literalToN3("foobar")));
 
       .arg(Node::literalToN3("foobar")));
  
</code>  
+
</syntaxhighlight>  
  
 
This will create the same query we used above only using no hard-coded components whatsoever.  
 
This will create the same query we used above only using no hard-coded components whatsoever.  
Line 137: Line 137:
 
# qdbus org.kde.NepomukServer /nepomukserver \
 
# qdbus org.kde.NepomukServer /nepomukserver \
 
     org.kde.NepomukServer.quit
 
     org.kde.NepomukServer.quit
</code>  
+
</syntaxhighlight>  
  
 
It can then be restarted by simply calling ''nepomukserver'' again. In many debugging situations it might be of interest to pipe the output of the server (and all services) to a file:  
 
It can then be restarted by simply calling ''nepomukserver'' again. In many debugging situations it might be of interest to pipe the output of the server (and all services) to a file:  
Line 143: Line 143:
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
 
# nepomukserver 2> /tmp/nepomuk.stderr
 
# nepomukserver 2> /tmp/nepomuk.stderr
</code>  
+
</syntaxhighlight>  
  
 
Also interesting to know is that Nepomuk defines a set of debugging areas for the services and the server itself. Use ''kdebugdialog'' to enable or disable them.  
 
Also interesting to know is that Nepomuk defines a set of debugging areas for the services and the server itself. Use ''kdebugdialog'' to enable or disable them.  
Line 155: Line 155:
 
# qdbus org.kde.NepomukServer /servicemanager \   
 
# qdbus org.kde.NepomukServer /servicemanager \   
 
     org.kde.nepomuk.ServiceManager.startService <servicename>
 
     org.kde.nepomuk.ServiceManager.startService <servicename>
</code>  
+
</syntaxhighlight>  
  
 
<br>  
 
<br>  
Line 173: Line 173:
 
connect( scm, SIGNAL(statementsRemoved()),
 
connect( scm, SIGNAL(statementsRemoved()),
 
         this, SLOT(slotStatementsRemoved()) );
 
         this, SLOT(slotStatementsRemoved()) );
</code>  
+
</syntaxhighlight>  
  
 
== Remove all Strigi-indexed data  ==
 
== Remove all Strigi-indexed data  ==
Line 185: Line 185:
 
   ?g <http://www.strigi.org/fields#indexGraphFor> ?r . }"`;
 
   ?g <http://www.strigi.org/fields#indexGraphFor> ?r . }"`;
 
   do nepomukcmd rmgraph "$a"; done
 
   do nepomukcmd rmgraph "$a"; done
</code>  
+
</syntaxhighlight>  
  
 
''This only works with sopranocmd from Soprano &gt;= 2.3.63!''  
 
''This only works with sopranocmd from Soprano &gt;= 2.3.63!''  
Line 198: Line 198:
 
PATH=/usr/lib/virtuoso:$PATH
 
PATH=/usr/lib/virtuoso:$PATH
 
export PATH
 
export PATH
</code>
+
</syntaxhighlight>

Revision as of 21:53, 29 June 2011


Contents

Development/Tutorials/Metadata/Nepomuk/TipsAndTricks


Nepmuk Tips and Tricks
Tutorial Series   Nepomuk
Previous   None
What's Next   n/a
Further Reading   Resource Handling with Nepomuk,

Advanced Queries with SPARQL, RDF and Ontologies in Nepomuk

Always initialize Nepomuk

Make sure that somewhere in the initialization code of your application or library Nepomuk is initialized via:

Nepomuk::ResourceManager::instance()->init(); </syntaxhighlight>


Using ontology URIs in your code

One often needs the URI of a specific class or a specific property in ones code. And not all ontologies are provided by the very convenient Soprano::Vocabulary namespace.

The solution is rather simple: create your own vocbulary namespaces by using Soprano's own onto2vocabularyclass command line tool. It can generate convenient vocabulary namespaces for you. The Soprano documentation shows how to use it manually or even simpler with a simple CMake macro.


Mind the Difference between QString and QUrl

Nepomuk::Resource provides two constructors: one taking a QString as identifier or URI and one taking a QUrl.

The latter one is really simple: the given URI is used as the resource URI. If the resource exists, its data is used, otherwise it will be created with exactly that URI.

The QString one is a bit trickier. It will try to be clever about the parameter and see if it is a URI. If no resource with that URI (if it is a URI) exists, it is interpreted as an identifier (nao:identifier). Resource checks if a resource with that identifier exists. If so, its data is loaded, if not, a new resource with a random URI and that string as identifier is created.

However, be aware that nothing is written to Nepomuk until the first writing call to Resource such as setProperty or addType.


Debugging the created data

Using sopranocmd

When using Nepomuk one creates a lot of RDF statements in the Nepomuk RDF storage. It is often of interest to check which data has been created, if statements have been correctly created or simply look at existing data.

Soprano provides a nice command line client to do all this called sopranocmd. It provides all the features one needs to debug data: it can add and remove statements, list and query them, import and export whole RDF files, and even monitor for statementAdded and statementRemoved events.

To access the Nepomuk storage one would typically use the D-Bus interface:

# sopranocmd --dbus org.kde.NepomukStorage --model main <command> \
    <parameters>

If one wanted to list all the resources that have been tagged with the tag whose resource URI is nepomuk:/foobar one would use the following command:

# sopranocmd --dbus org.kde.NepomukStorage --model main list \
    "" "" "<nepomuk:/foobar>"

or one would use a SPARQL query (sopranocmd supports the standard URI prefixes out of the box):

# sopranocmd --dbus org.kde.NepomukStorage --model main query \
    "select ?r where { ?r nao:hasTag ?tag . \
                       ?tag nao:prefLabel 'foobar'^^xsd:string . }"
To monitor all statements that are added and removed from the Nepomuk storage one would simply use the following command (as with list one can specify a filter to only list the added and removed statements one is interested in):
# sopranocmd --dbus org.kde.NepomukStorage --model main monitor
# sopranocmd --help
is your friend for all details.

nepomukcmd

As a shortcut add the following to your .bashrc to avoid having to type in the dbus and model parameters all the time:

KDE 4.6 and beyond:

alias nepomukcmd="sopranocmd --socket `kde4-config --path socket`nepomuk-socket --model main --nrl"

before KDE 4.6:

alias nepomukcmd="sopranocmd --socket `kde4-config --localprefix`/share/apps/nepomuk/socket --model main --nrl"

(Be aware that the --nrl parameter is only available in Soprano 2.3.63 and above.)

Using Konqueror

In the Nepomuk playground repository lives a KIO slave which can handle the nepomuk:/ protocol. It will display all properties of a Nepomuk resource including its links to other resources and the backlinks. This is a convenient way of looking at the Nepomuk data. The KIO slave even support removal of resources.

Nepomuk kio slave.png


Using NepomukShell

NepomukShell is a maintenance and debugging tool, which lives in its own git repository at nepomukshell. It is a simple tool that let's one browse all resources in Nepomuk. Additionally it allows to create subclasses and properties (Caution: do only create subclasses and properties from PIMO classes and properties!) and remove resources.

Pimoshell.png

Constructing SPARQL queries

Hint: In most cases the Nepomuk Query API should be enough and prevent you from writing your own SPARQL which is hard to debug.

Whenever doing something a bit fancier with Nepomuk one has to use SPARQL queries via <code cppqt="cppqt"> Nepomuk::ResourceManager::instance()->mainModel()

   ->executeQuery( myQueryString, 
                   Soprano::Query::QueryLanguageSparql );

</syntaxhighlight> Constructing these queries can be a bit cumbersome since one has to use a lot of class and property URIs from different ontologies. Also literals have to be formatted according to the N3 syntax used in SPARQL. Luckily Soprano provides the necessary tools to do exactly that: Soprano::Node::toN3, Soprano::Node::resourceToN3, and Soprano::Node::literalToN3 take care of all formatting and percent-encoding you need. Using those methods the code to create queries might look ugly but the resulting queries are more likely to be correctly encoded and introduce less code duplication.

Typically one would use QString::arg like so (be aware that the standard prefixes are NOT supported out-of-the-box as with sopranocmd):

<code cppqt="cppqt"> using namespace Soprano;

QString myQuery

    = QString("select ?r where { "
              "?r %1 ?v . "
              "?v %2 %3 . }")
      .arg(Node::resourceToN3(Vocabulary::NAO::hasTag()))
      .arg(Node::resourceToN3(Vocabulary::NAO::prefLabel()))
      .arg(Node::literalToN3("foobar")));

</syntaxhighlight>

This will create the same query we used above only using no hard-coded components whatsoever.

Restarting Nepomuk and its Services

The Nepomuk services are controlled by the nepomukserver application which is started on KDE login. The nepomukserver will take care of starting and stopping all services.

It is possible to stop the server and all services alltogether by simply calling a D-Bus method:

# qdbus org.kde.NepomukServer /nepomukserver \
    org.kde.NepomukServer.quit

It can then be restarted by simply calling nepomukserver again. In many debugging situations it might be of interest to pipe the output of the server (and all services) to a file:

# nepomukserver 2> /tmp/nepomuk.stderr

Also interesting to know is that Nepomuk defines a set of debugging areas for the services and the server itself. Use kdebugdialog to enable or disable them.

Or one can stop and start single services. In most cases this is sufficient since each service is run in its own process. Thus, changes to a service plugins will be picked up directly:

# qdbus org.kde.NepomukServer /servicemanager \   
    org.kde.nepomuk.ServiceManager.stopService <servicename>
 
# qdbus org.kde.NepomukServer /servicemanager \   
    org.kde.nepomuk.ServiceManager.startService <servicename>


Listening to changes in the database

Nepomuk's database does only contain statements, i.e. quadruples. To date Soprano's Model does provide four signals which can be used to monitor new and removed statements: statementsAdded and statementsRemoved as well as their counterparts which have as paramter the added or removed statement.

It is recommended to use Soprano's SignalCacheModel when listening to changes to prevent a slowdown of the whole system as the signals are emitted for each statement:

<code cppqt="cppqt"> Soprano::Util::SignalCacheModel* scm = new Soprano::Util::SignalCacheModel(

   Nepomuk::ResourceManager::instance()->mainModel() );

connect( scm, SIGNAL(statementsAdded()),

        this, SLOT(slotStatementsAdded()) );

connect( scm, SIGNAL(statementsRemoved()),

        this, SLOT(slotStatementsRemoved()) );

</syntaxhighlight>

Remove all Strigi-indexed data

Strigi produces a lot of data in Nepomuk. There might be times where one wants to remove all that data manually.

The little command below removes all data created by Strigi (caution: this could take a long time):

for a in `nepomukcmd --foo query "select distinct ?g where { \
  ?g <http://www.strigi.org/fields#indexGraphFor> ?r . }"`;
  do nepomukcmd rmgraph "$a"; done

This only works with sopranocmd from Soprano >= 2.3.63!


Starting Nepomuk Sever from the Trunk in Ubuntu

Ubuntu packages virtuoso slightly differently. It provides a package called virtuoso-nepomuk which installs the executable virtuoso-t in the /usr/lib/virtuoso/ directory for security purposes.

When running Nepomuk from the trunk, the nepomukserver is unable to find the virtuoso-t executable, and therefore the NepomukStorage Service fails to initialize. One way to fix this is to adjust the PATH environment variable.

PATH=/usr/lib/virtuoso:$PATH
export PATH

KDE® and the K Desktop Environment® logo are registered trademarks of KDE e.V.Legal