Conditional Restrictions

[Buwujiu]

Replace script restrictions with a conditional restriction mechanism that can be represented in a Lectern Dictionary schema.

Documentation of the proposed meta-schema was submitted as part of the working PR for this change. You can review the docs here.

Tasks

• Make restrictions property an array of restrictions objects
• Move unique property out of restrictions and onto field
• Remove script restriction
• Create empty restriction
• Create compare restriction
• Add conditional restriction (if/else/then) structure to schema
• Create conditional restriction match rules:
  • codeList
  • compare
  • count
  • exists
  • range
  • regex
  • value
• Conditional restriction case rules:
  • case in an if for how many conditions need to be true to resolve then restrictions
  • case inside a condition that determines how many fields must pass the match condition's rules
  • arrayFieldCase inside a condition that determines how many values in an array field must match the condition's rules

Conditional Restrictions

Conditional restrictions are meant to replace script restrictions for all future versions of lectern dictionaries.

Script restrictions are being removed for several reasons, but most importantly is they can never be considered safe to execute arbitrary script validations from untrusted dictionaries. It is always risky to execute arbitrary code without first validating it, which would cause lectern dictionaries to not be easily shareable and reusable between projects and teams.

Some other considerations why scripts are being replaced include that script restrictions are:

• difficult to write
  • limits who can prepare Lectern dictionaries to those comfortable with programming
• difficult to read and interpret
  • makes it difficult to prepare data for validation
• prone to errors
  • could crash while executing script during data validation
  • could fail to halt, causing infinite validation time
  • impossible to validate by Lectern server
• poorly documented
• limited to JavaScript, restricting the environments that can implement lectern dictionary validators

In place of the script restriction, this PR proposes extending the field restrictions property to allow conditional restrictions. Conditional restrictions change the restrictions on a field based on the value of other fields in the same record.

A genomics related example of this might be where the field tumor_grading_system is required when the specimen has tumour_normal_designation value of tumor, but should be left empty in other cases. An example for a schema with just these two fields would be:

{
	"name": "specimen",
	"description": "Incomplete schema for specimen data; demonstrates conditional restriction",
	"fields": [
		{
			"name": "tumour_normal_designation",
			"valueType": "string",
			"restrictions": {
				"required": true,
				"codeList": ["Normal", "Tumour"]
			}
		},
		{
			"name": "percent_tumour_cells",
			"description": "Decimal value that represents the percent of tumour cells compared to the number of total cells in the specimen.",
			"meta": {
				"reference": "NCIT:C159484"
			},
			"valueType": "number",
			"restrictions": {
				"if": {
					"fields": ["tumour_normal_designation"],
					"match": {
						"value": "Tumour"
					}
				},
				"then": {
					"required": "true"
				},
				"else": {
					"empty": true
				}
			}
		}
	]
}

Other Changes to Dictionary Structure

• New restriction types: compare, empty