ShEx

Obsolete - please see the ShEx github wiki

The Shape Expressions (ShEx) language describes RDF graph structures. These descriptions identify predicates and their associated cardinalities and datatypes. ShEx shapes can be used to communicate data structures associated with some process or interface, generate or validate data, or drive user interfaces. It is intended to:

validate RDF documents.
communicate expected graph patterns for interfaces.
generate user interface forms and interface code.

A simple ShEx demo shows up data validates against a schema.

The ShEx Homepage includes links to specifications and examples. This wiki gathers extensions and conventions maintained by the ShEx community and provides a scratch space for discussions around advancing the specification.

If you would like to participate in the standardization of ShEx, please join the ShEx Community Group.

Example

<IssueShape> {                            # A Issue shape
    :state [ :unassigned :assigned ];     #   has a state with 2 possible values
    :reportedBy @<UserShape>;             #   is reported by a user
    :reportedOn xsd:date;                 #   is reported on a date
    ( :reproducedBy @<UserShape>          #   can optionally have 2 properties
    ; :reproducedOn xsd:date              #     reproducedBy/On
    )?;
    :related @<IssueShape>*               #   is related to several other issues
}

<UserShape> {                             # A user shape can have either
    ( foaf:name xsd:string                #  name or
    | foaf:givenName xsd:string+ ;        #  several given names and
      foaf:familyName xsd:string          #   family name
    ); 
    foaf:mbox shex:IRI ?                  # mbox Optional, any IRI
}

A separate page contains some simple examples using ShEx.

The following list contains a list of examples that employ ShEx:

Shex definition can be defined in three syntaxes: ShExC (ShEx Compact format), ShExJ (a JSON-LD syntax) and ShExR (the RDF corresponding to ShExJ).

Implementations

ShEx.js

Formats: ShExC, ShExJ, ShExR
Language: javascript based
Algorithm: Partition
Developer: Eric Prud'Hommeaux

Live Demo Examples:

simple validator interface

SHACLex

Syntax: ShExC, ShExJ, ShExR (ongoing work), SHACL
Developer: Jose Emilio Labra Gayo
Algorithm: Implementation based on monads and regular expression detivatives with support for both ShEx and SHACL
Programming language: Scala

Live demo:

SHACLEX server

ShEx Ruby

Syntax: ShExC, ShExJ, ShExR
Developer: Gregg Kellogg
Programming language: Ruby

ShEx implementation in Java (Lille)

Syntax: ShExJ
Developer: Iovka Boneva
Programming language: Java

Old implementations

RDFShape

Syntax: ShExc (Shex compact syntax) with some extensions like regex
Developer: Jose Emilio Labra Gayo
Algorithm: Regular expression derivatives
Programming language: Scala
Extra features: Online RDF validator based on Shexcala

Shexcala

Syntax: ShEx compact syntax
Developer: Jose Emilio Labra Gayo
Algorithm: Regular expression derivatives and backtracking (by selection)
Programming language: Scala
Extra features: Negation, Reverse arcs, language tags, regexps
Note: this library is no longer maintained. The new version is SHACLex

JSShexTest (ShEx1)

Formats: ShExR and ShExC(partly)
Language: Javascript based
Algorithm: State based
Developer: Jesse van Dam
Working version: [1].
Source code [2] (uses a local web server that can be started with ruby and accessed via localhost:4567).

For the validation code see [3] for the validation process (easy to read). Further description can be found here at ValidationCode.

Haws

Syntax: Abstract syntax (no parser)
Semantics: Closed shapes based on operational semantics
Developer: Jose Emilio Labra Gayo
Algorithm: Backtracking. This implementation was just a research prototype
Programming language: Haskell

Test cases

ShapeMap

The complete validation process is separated in 2 steps.

Step 1 The creation of the Node to (multiple) Shapes mapping named as the "ShapeMap" Shape Map using different techniques, which are at the moment not part of the SHEX standard. The only mapping included in the standard is the default "ShapeMap". This "ShapeMap" contain only a mapping between the start Node (given as parameter to validation process) and start Shape given in the ShEx schema.

The ShEx/ShapeMap page describes different techniques for associating nodes with shapes.

Step 2 The ShEx validation as described in the spec[4]. Every node in the "ShapeMap" is validated against each of the associated shapes. Any nodes that neither are in the ShapeMap nor visited during the validation process are not validated.

This separation ensures that shapes are semantically independent and can be linked to different things dependent on the node to shape mapping.

Syntax and Serialization formats

The following serialization formats exists

ShExc - Shex compact format as given in the example above
ShExR - The RDF version for ShEx, schema's herefore can be found at ShExR schema as ShExc, ShExR as ShExR, ShExR as ShExJ
ShExJ - The JSON-LD(= JSON) version for ShEx, which can be created from ShExR(data model is the same) after applying the context in @context and framing, ShExJ in JSON grammer
ShExV - The validation report format in JSON, validation rapport in JSON grammer

Publications about ShEx

Complexity and Expressiveness of ShEx for RDF, In International Conference on Database Theory (ICDT) 2015. With S. Staworko, J. E. Labra Gayo, S. Hym, E. G. Prud’hommeaux, and H. Solbrig. PDF
Towards an RDF validation language based on Regular Expression derivatives, Jose Emilio Labra Gayo, Eric Prud'Hommeaux, Slawek Staworko and Harold Solbrig. PDF Slides
Shape Expressions: An RDF validation and transformation language, Eric Prud'hommeaux, Jose Emilio Labra Gayo, Harold Solbrig, 10th International Conference on Semantic Systems, Sept. 2015, Leipzig, Germany, PDF Slides
Validating and Describing Linked Data Portals using RDF Shape Expressions, Jose Emilio Labra Gayo, Eric Prud'hommeaux, Harold Solbrig, 1st Workshop on Linked Data Quality, Sept. 2015, Leipzig, Germany, PDF Slides
An RDF validation and transformation language, Eric Prud'hommeax, Jose Emilio Labra Gayo and Harold Solbrig, SEM '14 Proceedings of the 10th International Conference on Semantic Systems, Leipzig, Germany, September 04 - 05, 2014 (Best Paper Award)

Discussion & Proposed Features

See here for the currently ongoing discussions

Issuelist 2.0

Issuelist 2.1

Submitting & resolving issues (under discussion)

All ShEx issues are tracked in Github. There is no requirement for submitting a new issue. Anyone, even people outside the ShEx CG can open an issue. However, we have a pre-defined workflow for closing issues.

After an issue is opened, it can be discussed directly in github, in form of other comments, or during the group telcos. People can suggest a proposal to close an issue by adding a comment starting with the keyword "PROPOSAL:" and explaining the proposed solution. The issue is tagged with the label "Voting-Needed" and an email is sent to the mailing list asking for all members to vote. All members can cast their votes directly on the proposal comment using the reaction icons. The suggested reactions are "+1", "-1" and "confused". The issue can be resolved if there are only +1 reactions 1 week after the proposal announcement or there is no competing proposal. In other cases, the issues are resolved during the groups telcos. Every issue is resolved with a new comment stating the accepted proposal, the member votes and a comment on if it was resolved automatically or during a telco.