UCO semi-open vocabularies need to become enumerations of `xsd:string`s

# Background

UCO gives sets of strings that suggest values to use for certain properties.  UCO has called this the "Semi-open vocabulary" design pattern: While certain strings should be used, strings not included in UCO's ontology files can still be used instead when the user knows the value they are using is not yet in UCO, and/or might never be in UCO for whatever reason (e.g., confidentiality of non-public vocabularies).

UCO uses a syntax for its semi-open vocabularies that uses what has appeared to date to be an OWL design pattern for defining a datatype as a set of typed string values.  However, it turns out the pattern used by UCO is invalid OWL syntax.  [Issue 593](https://github.com/ucoProject/UCO/issues/593) covers half of the issues raised by an externally-developed OWL syntax reviewing tool, [ROBOT](https://github.com/ontodev/robot)[^1].  Issue 593's correction has no known impact on user data.

The other half of the syntax correction takes this form, which unfortunately differs from UCO's user data guidance since UCO's earliest examples.  Given the known non-0 impact on user data, this is being treated as a non-fast-track proposal.

Here is (again, selected for its small size) the Bitness vocabulary, as implemented in UCO 1.3.0 and long before:

```turtle
vocabulary:BitnessVocab
	a rdfs:Datatype ;
	rdfs:label "Bitness Vocabulary"@en-US ;
	rdfs:comment "Defines an open-vocabulary of word sizes that define classes of operating systems."@en ;
	owl:equivalentClass [
		a rdfs:Datatype ;
		owl:onDatatype xsd:string ;
		owl:oneOf (
			"32"^^vocabulary:BitnessVocab
			"64"^^vocabulary:BitnessVocab
		) ;
	] ;
	.
```

Issue 593 cut the triple `[] owl:onDatatype xsd:string .`.  This Issue finishes the OWL syntax correction by changing the members to values typed only as `xsd:string` (which is the OWL-wide (IIRC, RDF-wide) default when no other type is stated):

```turtle
vocabulary:BitnessVocab
        a rdfs:Datatype ;
        rdfs:label "Bitness Vocabulary"@en-US ;
        rdfs:comment "Defines an open-vocabulary of word sizes that define classes of operating systems."@en ;
        owl:equivalentClass [
                a rdfs:Datatype ;
                owl:onDatatype xsd:string ;
                owl:oneOf (
                        "32"
                        "64"
                ) ;
        ] ;
        .
```

The syntax error this addresses is that the enumeration attempted to assign a datatype to the enumeration's member-literals; but, that datatype is not yet defined at the time the enumeration is being consumed by the parsing process, because they are inside the referenced datatype's definition.  ROBOT describes this as a cyclic reference in a definition, and calls it a syntax error.  The "Bug description" section below goes into the details describing why the cycle is also a symptom of another syntax error.


# Requirements


## Requirement 1

UCO vocabularies must adhere to OWL syntactic requirements for custom datatypes. 


## Requirement 2

Any UCO vocabulary `X` must not use itself as a datatype in its enumerated literals, due to OWL syntax requirements (cited in "Bug description" section).


# Risk / Benefit analysis


## Benefits

* Presently, OWL tools using the [OWL API library](https://github.com/owlcs/owlapi)[^1] declare UCO as having syntax errors (one per vocabulary) using the cyclic-reference pattern.  (The error's call stack ends in [this method](https://github.com/owlcs/owlapi/blob/2d5441826e30199813ff91ef2f27489967279a60/api/src/main/java/org/semanticweb/owlapi/profiles/violations/CycleInDatatypeDefinition.java#L47).)  From (offline) testing the patch with ROBOT, this change proposal addresses that error type, removing a barrier to compatibility for other OWL-based ontologies that might adopt UCO.
* Using plain `xsd:string` in vocabularies resolves a blocking design question on JSON-LD context dictionaries, on how to assign datatypes to semi-open vocabulary properties without inducing SHACL errors when a value outside the suggested set is used.
* This reduces programming languages' complexities in assigning datatypes to certain properties only when certain values are used.  (This is a concern downstream from the UCO ontology, and applies to e.g. Python bindings.)


## Risks

* All CASE and UCO data using semi-open vocabularies would need to revert vocabulary assignments to simple strings.  This would be a backwards-incompatible change because 1.3.0-and-prior data would be flagged as erroneous, but this change can be implemented in stages with the warning pattern used for other proposals with pre-2.0.0 and post-2.0.0 effects.


# Competencies demonstrated


## Competency 1

A hash is being migrated from UCO 1.3.0 to UCO 1.4.0.  The form in UCO 1.3.0 is:

```json
{
    "@context": {
        "kb": "http://example.org/kb/",
        "types": "https://ontology.unifiedcyberontology.org/uco/types/",
        "vocabulary": "https://ontology.unifiedcyberontology.org/uco/types/"
    },
    "@graph": {
        "@id": "kb:Hash-9ed55c42-3204-4e45-ace7-7ae6bd7d8f38",
        "@type": "types:Hash",
        "types:hashMethod": {
            "@type": "vocabulary:HashNameVocab",
            "@value": "SHA3-256"
        },
        "types:hashValue": {
            "@type": "xsd:hexBinary",
            "@value": "36f028580bb02cc8272a9a020f4200e346e276ae664e45ee80745574e2f5ab80"
        }
    }
}
```


### Competency Question 1.1

How is the hash supposed to be spelled with this proposal?


#### Result 1.1

Note that two lines are dropped: The `"vocabulary"` prefix from the context dictionary will never be needed by users, and `hashMethod` no longer needs the typed-literal JSON object in its spelling.

```json
{
    "@context": {
        "kb": "http://example.org/kb/",
        "types": "https://ontology.unifiedcyberontology.org/uco/types/"
    },
    "@graph": {
        "@id": "kb:Hash-9ed55c42-3204-4e45-ace7-7ae6bd7d8f38",
        "@type": "types:Hash",
        "types:hashMethod": "SHA3-256",
        "types:hashValue": {
            "@type": "xsd:hexBinary",
            "@value": "36f028580bb02cc8272a9a020f4200e346e276ae664e45ee80745574e2f5ab80"
        }
    }
}
```


### Competency Question 1.2

How is a hash using an algorithm name not in UCO supposed to be spelled?


#### Result 1.2

```turtle
# UCO 1.4.0 form
kb:Hash-9ed55c42-3204-4e45-ace7-7ae6bd7d8f38
	a types:Hash ;
	types:hashMethod "FooHash" ;
	types:hashValue "098f6bcd4621d373cade4e832627b4f6"^^xsd:hexBinary ;
	.
```


### Competency Question 1.3

How should semi-open vocabulary properties' ranges be spelled?


#### Result 1.3

These properties' ranges should continue to be spelled as the union of `xsd:string` and the UCO Vocabulary IRI.  The [OWL 2 Syntax document, Section 9.4](https://www.w3.org/TR/2012/REC-owl2-syntax-20121211/#Datatype_Definitions) gives an example for a restricted string-property.  UCO's semi-open vocabulary still permits general `xsd:string`, so an OWL Union'd range should be used in `owl:DatatypeProperty`s, and vocabulary membership should be reviewed with SHACL.


# Solution suggestion

No changes are necessary to the `owl:DatatypeProperty` OWL definitions currently using the semi-open vocabulary pattern, taking for example `types:hashMethod`:

```turtle
types:hashMethod
	a owl:DatatypeProperty ;
	rdfs:label "hashMethod"@en ;
	rdfs:comment "A particular cryptographic hashing method (e.g., MD5)."@en ;
	rdfs:range [
		a rdfs:Datatype ;
		owl:unionOf (
			vocabulary:HashNameVocab
			xsd:string
		) ;
	] ;
	.
```

The SHACL shapes for the semi-open vocabulary pattern have been coming as trios of property shapes.  They need to be adjusted as follows for the implementing next UCO SEMVER-minor release, so current data raises SHACL warnings instead of errors.  Note that the `sh:Info`-severity "gentle suggestion" shape is now the shape that bears the enumeration.

```turtle
types:Hash
	sh:property
		[
			sh:datatype xsd:string ;
			sh:message "As of UCO 1.4.0, the datatype to use for types:hashMethod should be xsd:string.  Not using xsd:string will be an error in UCO 2.0.0." ;
			sh:path types:hashMethod ;
			sh:severity sh:Warning ;
		] ,
		[
			sh:maxCount "1"^^xsd:integer ;
			sh:minCount "1"^^xsd:integer ;
			sh:nodeKind sh:Literal ;
			sh:path types:hashMethod ;
		] ,
		[
			sh:message "Value is not member of the vocabulary HashNameVocab." ;
			sh:in (
				"MD5"
				"MD6"
				"SHA1"
				"SHA224"
				"SHA256"
				"SHA3-224"
				"SHA3-256"
				"SHA3-384"
				"SHA3-512"
				"SHA384"
				"SHA512"
				"SSDEEP"
			) ;
			sh:path types:hashMethod ;
			sh:severity sh:Info ;
		]
		;
	.
```

(An aside: the new `sh:Warning` shape makes some new cases that previously failed now pass as warnings, like using an integer for `hashMethod`'s value.  Given the warnings, this feels like a low-risk relaxation.)

For UCO 2.0.0, the first two shapes would combine, leaving the implementation as follows.

```turtle
types:Hash
	sh:property
		[
			sh:datatype xsd:string ;
			sh:maxCount "1"^^xsd:integer ;
			sh:minCount "1"^^xsd:integer ;
			sh:nodeKind sh:Literal ;
			sh:path types:hashMethod ;
		] ,
		[
			sh:message "Value is not member of the vocabulary HashNameVocab." ;
			sh:in (
				"MD5"
				"MD6"
				"SHA1"
				"SHA224"
				"SHA256"
				"SHA3-224"
				"SHA3-256"
				"SHA3-384"
				"SHA3-512"
				"SHA384"
				"SHA512"
				"SSDEEP"
			) ;
			sh:path types:hashMethod ;
			sh:severity sh:Info ;
		]
		;
	.
```

Due to a processing efficiency issue, one final modification from current practice is suggested: The "Gentle suggestion" property shape should change from an inlined, blank-node-identified `sh:PropertyShape` to a class-independent, IRI-identified `sh:NodeShape` that uses the targeting statement `... sh:targetObjectsOf types:hashMethod .`  The reason for this is that certain SHACL reporting practices, like those used in UCO's unit testing, inline all blank node property shapes in the SHACL result-sets.  For some vocabularies, especially longer ones that suggest intensive extension (in particular, the 200+-member `vocabulary:ObservableObjectRelationshipVocab`), this could be hopelessly obscuring in the result sets whenever users engage in local vocabulary extension.

This pattern is suggested for the implementation instead, to offer users (1) a reference to an IRI they can look up on the documentation site instead of getting an inlined and long anonymous shape, and (2) the opportunity to opt out of any specific "Gentle suggestion" shapes by using `sh:deactivate`:

```turtle
types:hashMethod-objects-in-shape
	a sh:NodeShape ;
	sh:in (
		"MD5"
		"MD6"
		"SHA1"
		"SHA224"
		"SHA256"
		"SHA3-224"
		"SHA3-256"
		"SHA3-384"
		"SHA3-512"
		"SHA384"
		"SHA512"
		"SSDEEP"
	) ;
	sh:message "Value is not member of the vocabulary HashNameVocab." ;
	sh:severity sh:Info ;
	sh:targetObjectsOf types:hashMethod ;
	.
```

The replacement for the class's `sh:property` links would be as follows, using an `rdfs:seeAlso` to preserve some reference on the documentation website without any other mechanical impact:

```patch
 types:Hash
+	rdfs:seeAlso types:hashMethod-objects-in-shape ;
 	sh:property
 		[
 			sh:datatype xsd:string ;
 			sh:maxCount "1"^^xsd:integer ;
 			sh:minCount "1"^^xsd:integer ;
 			sh:nodeKind sh:Literal ;
 			sh:path types:hashMethod ;
- 		] ,
- 		[
- 			sh:message "Value is not member of the vocabulary HashNameVocab." ;
- 			sh:in (
- 				"MD5"
- 				"MD6"
- 				"SHA1"
- 				"SHA224"
- 				"SHA256"
- 				"SHA3-224"
- 				"SHA3-256"
- 				"SHA3-384"
- 				"SHA3-512"
- 				"SHA384"
- 				"SHA512"
- 				"SSDEEP"
- 			) ;
- 			sh:path types:hashMethod ;
- 			sh:severity sh:Info ;
 		]
 		;
 	.
```

If a user wishes to opt out of the gentle suggestions, they can add this triple in their extension SHACL shapes:

```
types:hashMethod-objects-in-shape
	sh:deactivated true ;
	.
```


# Bug description

While at least one tool reports that the current UCO vocabulary pattern is a cyclic definition error, the OWL syntax specification and RDF mapping documents together describe the problem as an inappropriate use of an OWL structure, `DatatypeDefinition`.

Looking through the [OWL 2 Mapping to RDF](https://www.w3.org/TR/2012/REC-owl2-mapping-to-rdf-20121211/) document, Table 16 includes this mapping:

IF this pattern is in the RDF graph *G*:

```
*:x owl:equivalentClass y .
{ DR(*:x) ≠ ε amd DR(y) ≠ ε }
```

THEN the following axiom is added to the ontology *OG*:

```
DatatypeDefinition( DR(*:x) DR(y) )
```

For quick reference, the smbols above mean:

* `*:x` is an IRI
* `y` is either a blank node or an IRI
* `DR(z)` is a function mapping a node `z` to a `DataRange` structre in the ontology *OG*
   - Nodes for `DR` can be IRIs or blank nodes
* `DR(z) ≠ ε` states that `DR(z)` is defined (or equivalently, the function at the value `z` is not empty)

Taken together, `owl:equivalentClass` as a predicate between two certain nodes pertaining to `DataRange`s imply a `DatatypeDefinition`.

`*:x` being in a `DatatypeDefinition` imposes a certain constraint on `*:x`, with which UCO has been incongruous to date.  [Section 9.4 of the OWL 2 Syntax document](https://www.w3.org/TR/2012/REC-owl2-syntax-20121211/#Datatype_Definitions) defines that in the structure `DatatypeDefinition ( DT DR )`, the `Datatype` that is the first parameter (`DT`, or `*:x`, or `uco-vocabulary:BitnessVocab`) is a "synonym for `DR`..."  And: "The datatypes defined by datatype definition axioms ... have empty lexical spaces and therefore they *must not* occur in literals."

The example in the OWL 2 syntax document that immediately follows that quote illustrates a datatype `a:SSN`, which is part of a datatype definition that would be spelled like this in Turtle:

```turtle
# First axiom: "Declaration( Datatype( a:SSN ) )"
a:SSN
	a rdfs:Datatype ;
	.

# Second axiom: "DatatypeDefinition( ..."
a:SSN
	owl:equivalentClass [
		a rdfs:Datatype ;
		owl:onDatatype xsd:string ;
		owl:withRestrictions (
			[
				xsd:pattern "[0-9]{3}-[0-9]{2}-[0-9]{4}" ;
			]
		) ;
	] ;
	.

# Third axiom: "DataPropertyRange( a:hasSSN a:SSN )"
a:hasSSN
	rdfs:range a:SSN ;
	.
```

Note the end of the example block states "there can be no literals of datatype *a:SSN*".

In UCO's case, there can be no literals of datatype `vocabulary:BitnessVocab`, `vocabulary:HashNameVocab`, etc.


# Coordination

- Tracking in Jira ticket [OCUCO-323](https://cyberontology.atlassian.net/browse/OCUCO-323)
- [x] Administrative review completed, proposal announced to Ontology Committees (OCs) on 2024-08-12
- [x] Requirements to be discussed in OC meeting, [2024-08-20](https://cyberontology.atlassian.net/wiki/spaces/CDOC/pages/2248212489/2024-08-20+CDO+Ontology+Committees)
- [x] Requirements Review vote occurred, passing, on 2024-08-20
- [x] *Requirements development phase completed.*
- [x] Solution announced to OCs on 2025-02-26
- [x] Solutions Approval to be discussed in OC meeting, [2025-03-11](https://cyberontology.atlassian.net/wiki/spaces/CDOC/pages/2489253889/)
- [x] Solutions Approval vote occurred, passing, on 2025-03-11
- [x] *Solutions development phase completed.*
- [x] [Backwards-compatible implementation](https://github.com/ucoProject/UCO/pull/630) merged into `develop` for the next release
- [x] [CASE backwards-compatible implementation](https://github.com/casework/CASE/pull/172) merged into `develop` for the next release
- [x] `develop` state with backwards-compatible implementation [merged](https://github.com/ucoProject/UCO/pull/653) into `develop-2.0.0`
- [x] [Backwards-incompatible implementation](https://github.com/ucoProject/UCO/pull/650) merged into `develop-2.0.0` (or *N/A*)
- [x] [CASE backwards-incompatible implementation](https://github.com/casework/CASE/pull/174) merged into `develop-2.0.0`
- [x] Milestone linked
- [x] Documentation [logged](https://github.com/ucoProject/ucoproject.github.io/commit/c19060c09efdd36b04e41b8c866602d50549dc79) in pending UCO release page
- [x] Documentation [logged](https://github.com/casework/casework.github.io/pull/325/commits/0d7bd267d318ed25b459e0fdae16bb7822d76969) in pending CASE release page
- [x] _Prerelease publication_: CASE `develop` branch [updated](https://github.com/casework/CASE/pull/172) to track UCO's updated `develop` branch
- [x] _Prerelease publication_: CASE `develop-2.0.0` branch [updated](https://github.com/casework/CASE/pull/174) to track UCO's updated `develop-2.0.0` branch

[^1]: Disclaimer: Participation by NIST in the creation of the documentation of mentioned software is not intended to imply a recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that any specific software is necessarily the best available for the purpose.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

UCO semi-open vocabularies need to become enumerations of `xsd:string`s #629

Background

Requirements

Requirement 1

Requirement 2

Risk / Benefit analysis

Benefits

Risks

Competencies demonstrated

Competency 1

Competency Question 1.1

Result 1.1

Competency Question 1.2

Result 1.2

Competency Question 1.3

Result 1.3

Solution suggestion

Bug description

Coordination

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

UCO semi-open vocabularies need to become enumerations of xsd:strings #629

Description

Background

Requirements

Requirement 1

Requirement 2

Risk / Benefit analysis

Benefits

Risks

Competencies demonstrated

Competency 1

Competency Question 1.1

Result 1.1

Competency Question 1.2

Result 1.2

Competency Question 1.3

Result 1.3

Solution suggestion

Bug description

Coordination

Footnotes

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

UCO semi-open vocabularies need to become enumerations of `xsd:string`s #629