SWObjects Source Code Directory

This is a set of utilities implemented in C++ for working with SemWeb objects (triples, graphs, query patterns, SPARQL and N3 operations, etc). This implements query mapping based on Mapping Rules and provides the base for a SPASQL engine in MySQL.

• source tarball: SWObjects_0.1.tar.gz
• linux libstdc++6 executable: linux-libstdc++6/SWtransformer
• windows executable: win/SWtransformer.exe

For more information on SPASQL: ① SPASQL Example · ② SPASQL-MySQL · ③ SPASQL XTech Paper · ④ XTech Slides · ⑤ SPASQL CVS Tree

Table of Contents

• Building SWObjects
• Files
  • Building and Testing
  • Core
  • Parsers
• Architecture
• Sample Compilation
• Issues
• Apache Directory Listing

Building SWObjects

The SWObjects library and executables can be built from the source in the tarball. This package includes one directory, SWObjects_0.1; running make test from this directory will build and test the executables. Building the parsers requires bison 2.3 or flex 2.5 but you may use the pre-build parsers by adding an include directory to the compilation: INCLUDES=-Iwin make test. Cygwin's version of make requires an extra environment variable as well: PWD=. make test (note, these can be combined). The Makefile compilation has been tested on linux, OSX and Windows/Cygwin (windows users may also use the MSVC project). Below is a sample compilation.

The executable tests/execute_HealthCare1 (which will be renamed soon and place in a bin/ directory) serves as a general interface to SWObjects query mapping functionality. It takes as an argument the name of a file containing a SPARQL SELECT query. With no other input, they query is validated and printed out in a normalized form.

./test/execute_HealthCare1 tests/query_HealthCare1.rq

execute_HealthCare1 accepts any number of files containing SPARQL CONSTRUCTs which are used to tranform the SELECT query according to a Query Mapping Algorithm.

./test/execute_HealthCare1 tests/query_HealthCare1.rq tests/ruleMap_HealthCare1.rq

execute_HealthCare1 also accepts -s and -b flags to set the stem or base URI. The base URI is used to resolve relative URIs in parsing input. It defauts to <http://example.org/>. The stem URI, if provided, allowes execute_HealthCare1 to generate an SQL query based on the input SELECT query, transformed by any supplied CONSTRUCT rules.

./test/execute_HealthCare1 -s http://someClinic.exampe/DB/ tests/query_HealthCare1.rq

The mapping from a SPARQL query to an SQL query is described in Stem Query Mapping.

Files

The directory structure of the SWObjects Library is intentionally flat, as it is expected to be linked to other libraries to support web access, persistent storage, and other functionalities.

Building and Testing

• Makefile — targets for libSWObjects.a and sample executables.
• SWObjects.vcproj and SWObjects.sln — MSVC++ build script.
• tests/test_HealthCare1.cpp — query-mapping test to link to libSWObjects.a. Works with:
  • tests/query_HealthCare1.rq — simple example HCLS COI query.
  • tests/ruleMap_HealthCare1.rq — simple example HCLS COI rule-map.
  • tests/query_stem_Person.rq — simple example HCLS COI query to access Helen's data.

Core

• SWObjects.cpp and SWObjects.hpp — data structures to represent graph patterns for SPARQL and N3.
• RdfDB.cpp and RdfDB.hpp — implementation of a simple RDF database with multiple named graphs.
• RdfQueryDB.cpp and RdfQueryDB.hpp — overload of RdfDB which performs queries on graph patterns.
• ResultSet.cpp and ResultSet.hpp — simple SPARQL result set data structure.
• SWObjectDuplicator.hpp — deep-copy of a SPARQL compile tree (RDF Atoms allocated from a POSFactory.
• QueryMapper.hpp — overload of SWObjectDuplicator which maps queries backwards through deduction rules.
• RuleInverter.hpp — overload of SWObjectDuplicator which reverses SPARQL CONSTRUCT queries to transform queries acting on unmaterialized views; used by QueryMapper.
• SQLizer.hpp — overload of SWObjectDuplicator which maps from stem queries to SQL queries.
• POS2BGPMap.hpp — a query tree walker which gathers statistics about the use of POSs; used by QueryMapper and SQLizer.
• SPARQLSerializer.hpp — serialize SWObject tree as a SPARQL query.
• XMLSerializer.hpp — simple XML serializer interface.
• XMLQueryExpressor.hpp — simple XML representation of a SPARQL query.
• exs.hpp — simple stack trace exception generator to aid in development.
• common.hpp — common header file to streamline compilation.

Parsers

• ParserCommon.cpp and ParserCommon.hpp — common Driver source for all RDF-related parsers.
• SPARQLfedParser/SPARQLfedParser.ypp, SPARQLfedScanner.hpp and SPARQLfedScanner.lpp — parser for the SPARQLfed language; generates:
  • SPARQLfedParser/SPARQLfedParser.hpp - tokens passed from scanner to parser.
  • SPARQLfedScanner.cpp - lexical lex-style scannar.
  • SPARQLfedParser/SPARQLfedParser.cpp - LALR(1) parser tables and semantic actions.
  • SPARQLfedParser/location.hh, SPARQLfedParser/position.hh and SPARQLfedParser/stack.hh — auxilliary parser files created by bison.
• TurtleSParser/TurtleSParser.ypp, TurtleSScanner.hpp and TurtleSScanner.lpp — parser for the Turtle language; generates:
  • TurtleSParser/TurtleSParser.hpp - tokens passed from scanner to parser.
  • TurtleSScanner.cpp - lexical lex-style scannar.
  • TurtleSParser/TurtleSParser.cpp - LALR(1) parser tables and semantic actions.
  • TurtleSParser/location.hh, TurtleSParser/position.hh and TurtleSParser/stack.hh — auxilliary parser files created by bison.
• win/FlexLexer.h — supplementary flex file for systems without bison and flex.

Architecture

Matching BasicGraphPatterns

Given a set of TriplePatterns to match against a set of triples (which are also implemented as TriplePatterns), each atom (

Matching GraphPatterns

URI, BNode, Literal and TriplePattern Atoms

The primitive parts of speach (class: POS) and (class: TriplePattern) are created only once-each by a POSFactory. Applications typically create such a factory at the beginning of execution and destroy it at the end:

int main(int argc,char** argv) {
    w3c_sw::POSFactory posFactory;
    w3c_sw::SPARQLfedDriver sparqlParser("", &posFactory);
    ...
}

Storage

An RdfDB contains a DefaultGraphPattern and a set of NamedGraphPatterns, each of which contains TriplePatterns. This is more general than a strict RDF store in that the TriplePattern data structure allows for triples with variables in either the subject, predicate or object position.

Queries

The structure of queries is informed largely by the compile tree produced by a SPARQL parser, after it has simpified the tree. A typical query will construct a Select comprised of a WhereClause, which in turn holds a TableOperation and a BindingClause. TableOperations contain Filters and some, like TableConjunction and OptionalGraphPattern, contain other graph patterns. This TableOperation tree terminates in DefaultGraphPatterns and NamedGraphPattern, which contain the TriplePatterns described in Storage.

Sample Compilation

Following is an annotated sample build. The command you have to type are highlighted:

Get the tarball, either by web browser or a command-line tool like wget.

eric@mouni:/tmp/t$ wget http://www.w3.org/2008/04/SPARQLfed/SWObjects_0.1.tar.gz
--2008-09-07 18:07:56--  http://www.w3.org/2008/04/SPARQLfed/SWObjects_0.1.tar.gz
Resolving www.w3.org... 127.0.0.1
Connecting to www.w3.org|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 225799 (221K) [application/x-gzip]
Saving to: `SWObjects_0.1.tar.gz'

100%[======================================>] 225,799     --.-K/s   in 0.001s  

2008-09-07 18:09:36 (149 MB/s) - `SWObjects_0.1.tar.gz' saved [225799/225799]

Expand the tarball (for instance, with winzip).

eric@mouni:/tmp/t$ tar xzf SWObjects_0.1.tar.gz
eric@mouni:/tmp/t$ cd SWObjects_0.1/

build the source tree and test it.

eric@mouni:/tmp/t/SWObjects_0.1$ make test

builds dependencies...

touch SWObjects.d
makedepend -y -f SWObjects.d SWObjects.cpp -DYYTEXT_POINTER=1 -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser 2>/dev/null
touch ResultSet.d
makedepend -y -f ResultSet.d ResultSet.cpp -DYYTEXT_POINTER=1 -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser 2>/dev/null
touch RdfQueryDB.d
makedepend -y -f RdfQueryDB.d RdfQueryDB.cpp -DYYTEXT_POINTER=1 -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser 2>/dev/null
touch RdfDB.d
makedepend -y -f RdfDB.d RdfDB.cpp -DYYTEXT_POINTER=1 -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser 2>/dev/null
touch ParserCommon.d
makedepend -y -f ParserCommon.d ParserCommon.cpp -DYYTEXT_POINTER=1 -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser 2>/dev/null

builds parsers...

flex -o TurtleSScanner.cpp  TurtleSScanner.lpp
touch TurtleSScanner.d
makedepend -y -f TurtleSScanner.d TurtleSScanner.cpp -DYYTEXT_POINTER=1 -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser 2>/dev/null
flex -o SPARQLfedScanner.cpp  SPARQLfedScanner.lpp
touch SPARQLfedScanner.d
makedepend -y -f SPARQLfedScanner.d SPARQLfedScanner.cpp -DYYTEXT_POINTER=1 -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser 2>/dev/null
bison -o TurtleSParser/TurtleSParser.cpp TurtleSParser/TurtleSParser.ypp
sed -i~ 's,# define PARSER_HEADER_H,#pragma once,' TurtleSParser/TurtleSParser.hpp
touch TurtleSParser/TurtleSParser.d
makedepend -y -f TurtleSParser/TurtleSParser.d TurtleSParser/TurtleSParser.cpp -DYYTEXT_POINTER=1 -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser 2>/dev/null
bison -o SPARQLfedParser/SPARQLfedParser.cpp SPARQLfedParser/SPARQLfedParser.ypp
sed -i~ 's,# define PARSER_HEADER_H,#pragma once,' SPARQLfedParser/SPARQLfedParser.hpp
touch SPARQLfedParser/SPARQLfedParser.d
makedepend -y -f SPARQLfedParser/SPARQLfedParser.d SPARQLfedParser/SPARQLfedParser.cpp -DYYTEXT_POINTER=1 -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser 2>/dev/null

builds core modules...

g++ -DYYTEXT_POINTER=1  -g -W -Wall -Wextra -Wnon-virtual-dtor -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser -pipe    -c -o SPARQLfedParser/SPARQLfedParser.o SPARQLfedParser/SPARQLfedParser.cpp
g++ -DYYTEXT_POINTER=1  -g -W -Wall -Wextra -Wnon-virtual-dtor -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser -pipe    -c -o TurtleSParser/TurtleSParser.o TurtleSParser/TurtleSParser.cpp
g++ -DYYTEXT_POINTER=1  -g -W -Wall -Wextra -Wnon-virtual-dtor -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser -pipe    -c -o SPARQLfedScanner.o SPARQLfedScanner.cpp
g++ -DYYTEXT_POINTER=1  -g -W -Wall -Wextra -Wnon-virtual-dtor -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser -pipe    -c -o TurtleSScanner.o TurtleSScanner.cpp
g++ -DYYTEXT_POINTER=1  -g -W -Wall -Wextra -Wnon-virtual-dtor -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser -pipe    -c -o ParserCommon.o ParserCommon.cpp
g++ -DYYTEXT_POINTER=1  -g -W -Wall -Wextra -Wnon-virtual-dtor -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser -pipe    -c -o RdfDB.o RdfDB.cpp
g++ -DYYTEXT_POINTER=1  -g -W -Wall -Wextra -Wnon-virtual-dtor -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser -pipe    -c -o RdfQueryDB.o RdfQueryDB.cpp
g++ -DYYTEXT_POINTER=1  -g -W -Wall -Wextra -Wnon-virtual-dtor -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser -pipe    -c -o ResultSet.o ResultSet.cpp
g++ -DYYTEXT_POINTER=1  -g -W -Wall -Wextra -Wnon-virtual-dtor -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser -pipe    -c -o SWObjects.o SWObjects.cpp

builds library...

ar rcvs libSWObjects.a SPARQLfedParser/SPARQLfedParser.o TurtleSParser/TurtleSParser.o SPARQLfedScanner.o TurtleSScanner.o ParserCommon.o RdfDB.o RdfQueryDB.o ResultSet.o SWObjects.o
a - SPARQLfedParser/SPARQLfedParser.o
a - TurtleSParser/TurtleSParser.o
a - SPARQLfedScanner.o
a - TurtleSScanner.o
a - ParserCommon.o
a - RdfDB.o
a - RdfQueryDB.o
a - ResultSet.o
a - SWObjects.o

builds test...

g++ -DYYTEXT_POINTER=1  -g -W -Wall -Wextra -Wnon-virtual-dtor -I/tmp/t/SWObjects_0.1   -I/tmp/t/SWObjects_0.1/SPARQLfedParser -I/tmp/t/SWObjects_0.1/TurtleSParser -pipe  -o tests/execute_HealthCare1 tests/test_HealthCare1.cpp -L/tmp/t/SWObjects_0.1 -lSWObjects

(these warnings indicate parts of the code that aren't done.)

In file included from /tmp/t/SWObjects_0.1/QueryMapper.hpp:10,
                 from tests/test_HealthCare1.cpp:8:
/tmp/t/SWObjects_0.1/RuleInverter.hpp:170:2: warning: #warning not including dependent optionals yet
In file included from tests/test_HealthCare1.cpp:10:
/tmp/t/SWObjects_0.1/SQLizer.hpp:665: warning: unused parameter ‘p_SolutionModifier’
/tmp/t/SWObjects_0.1/SQLizer.hpp: In member function ‘virtual w3c_sw::Create* w3c_sw::SQLizer::create(w3c_sw::e_Silence, w3c_sw::URI*)’:
/tmp/t/SWObjects_0.1/SQLizer.hpp:732: warning: suggest braces around empty body in an ‘if’ statement
/tmp/t/SWObjects_0.1/SQLizer.hpp: In member function ‘virtual w3c_sw::Drop* w3c_sw::SQLizer::drop(w3c_sw::e_Silence, w3c_sw::URI*)’:
/tmp/t/SWObjects_0.1/SQLizer.hpp:737: warning: suggest braces around empty body in an ‘if’ statement

execute test with a sample query and a rule map...

tests/execute_HealthCare1 tests/query_HealthCare1.rq tests/ruleMap_HealthCare1.rq -s http://someClinic.exampe/DB/ > HealthCare1_test.results  && cat HealthCare1_test.results

output is the query transformed for the antecedent of the rule...

post-rule query (SPARQL):
SELECT ?somePerson 
WHERE
{
  {
    {
      _:ADMINISTRATION <http://myCo.exampe/DB/Patient#name> _:PATIENT_name .
    }
    {
      _:PATIENT_name <http://myCo.exampe/DB/Names#patient> ?someTest .
      ?someTest <http://myCo.exampe/DB/bar#attr2> ?FOOP_gen0 .
    }
  }
UNION
  {
    {
      _:ADMINISTRATION <http://myCo.exampe/DB/Patient#name> _:PATIENT_name .
    }
    {
      _:PATIENT_name <http://myCo.exampe/DB/Names#patient> ?somePerson .
      ?somePerson <http://myCo.exampe/DB/bar#attr2> ?FOOP_gen0 .
    }
  }
}

and transformed again into SQL...

Transformed query: 
SELECT union1.somePerson AS somePerson
       FROM (
    SELECT 1 AS _DISJOINT_, NULL AS somePerson, ADMINISTRATION.id AS ADMINISTRATION, ADMINISTRATION.name AS PATIENT_name, someTest.attr2 AS FOOP_gen0
           FROM Patient AS ADMINISTRATION
                INNER JOIN Names AS PATIENT_name ON PATIENT_name.id=ADMINISTRATION.name
                INNER JOIN bar AS someTest ON someTest.id=PATIENT_name.patient
  UNION
    SELECT 2 AS _DISJOINT_, PATIENT_name.patient AS somePerson, ADMINISTRATION.id AS ADMINISTRATION, ADMINISTRATION.name AS PATIENT_name, somePerson.attr2 AS FOOP_gen0
           FROM Patient AS ADMINISTRATION
                INNER JOIN Names AS PATIENT_name ON PATIENT_name.id=ADMINISTRATION.name
                INNER JOIN bar AS somePerson ON somePerson.id=PATIENT_name.patient
             ) AS union1

Issues

• smart_ptrs — is the overhead of keeping a POSFactory worth the performance windfall? It makes memory management more challenging. In general, should SWObjects use more copying? Should they instead use smart pointers?
• ser_union — SPARQLSerializer current fails to bound UNIONs in {}.

Apache Directory Listing

	Name	Last modified	Size	Description
	Parent Directory		-
	HEADER.html	2008-11-04 15:12	24K
	Makefile	2009-01-16 11:29	5.3K
	POS2BGPMap.hpp	2008-12-05 00:39	19K
	ParserCommon.cpp	2008-10-10 11:52	1.5K
	ParserCommon.hpp	2008-11-04 15:11	4.8K
	QueryMapper.hpp	2008-12-05 00:39	3.8K
	RdfDB.cpp	2008-12-05 00:39	1.6K
	RdfDB.hpp	2008-12-05 00:39	1.7K
	RdfQueryDB.cpp	2008-10-24 10:57	356
	RdfQueryDB.hpp	2008-12-05 00:39	6.8K
	ResultSet.cpp	2008-12-02 03:36	6.9K
	ResultSet.hpp	2008-12-02 04:55	5.5K
	RuleInverter.hpp	2008-12-05 00:39	21K
	SPARQLSerializer.hpp	2008-12-29 23:17	15K
	SPARQLfedParser/	2009-10-13 14:19	-
	SPARQLfedScanner.hpp	2008-10-03 07:06	2.3K
	SPARQLfedScanner.lpp	2008-10-03 07:06	18K
	SQLizer.hpp	2008-12-29 23:17	44K
	SWObjectDuplicator.hpp	2008-12-29 23:17	19K
	SWObjects.cpp	2008-12-29 23:17	31K
	SWObjects.hpp	2008-12-29 23:17	61K
	SWObjects.sln	2008-12-06 14:48	1.9K
	SWObjects_0.1.tar.gz	2008-10-24 10:59	186K
	TurtleSParser/	2009-10-13 14:19	-
	TurtleSScanner.hpp	2008-10-03 07:06	2.3K
	TurtleSScanner.lpp	2008-10-03 07:06	13K
	XMLQueryExpressor.hpp	2008-12-29 23:17	14K
	XMLSerializer.hpp	2008-08-26 05:30	2.4K
	common.hpp	2008-08-26 05:30	749
	exs.hpp	2008-11-04 15:12	2.1K
	location.hpp	2008-08-27 02:21	3.8K
	position.hpp	2008-08-27 02:21	3.6K
	stack.hpp	2008-08-27 02:21	2.9K
	tests/	2009-10-13 14:19	-
	win/	2009-10-13 14:19	-