Definitions

Restrictions

This package reserves JSON and TOML key $ref for references.

Configuration

A configuration is defined as

• a mapping with string keys (JSON strings and TOML strings) to represent

  • JSON objects and

  • TOML tables;

• an array to represent

  • JSON arrays and

  • TOML arrays;

• or other types (leafs) to represent

  • number, string, false, null and true JSON values and

  • string, integer, float, boolean, date-time, date and time TOML values.

The mapping values and array elements are again configuration instances.

Python implementation: rconf.Value.

Pointers

A JSON or TOML pointer identifies a value within a JSON or TOML document respectively.

JSON pointers

are defined in RFC 6901: JSON Pointer.

TOML pointers

are defined here as TOML keys, with the addition of array indices as defined for JSON pointers [1] .

Python implementation: rconf.pointer.Pointer.

References

An object or table (mapping) with a $ref key will be substituted by the value its string value references, compatible with the JSON Reference draft.

The reference JSON string or TOML string value is a URL with an optional fragment identifier [2] that follows the representation of the referent document: a JSON or TOML pointer.

Reference resolution is applied depth-first.

Example

A configuration with references to a TOML document (with TOML pointer) and a JSON document (with JSON pointer):

TOML

[pypa_build]
"$ref" = "https://raw.githubusercontent.com/pypa/build/main/pyproject.toml#project.description"

[icon]
"$ref" = "https://github.com/manifest.json#/icons/0"

JSON

{
    "pypa_build": {
        "$ref": "https://raw.githubusercontent.com/pypa/build/main/pyproject.toml#project.description"
    },
    "icon": {
        "$ref": "https://github.com/manifest.json#/icons/0"
    }
}

INI

[pypa_build]
$ref = https://raw.githubusercontent.com/pypa/build/main/pyproject.toml#project.description

[icon]
$ref = https://github.com/manifest.json#/icons/0

or more compact with implicitly created TOML tables,

TOML

pypa_build."$ref" = "https://raw.githubusercontent.com/pypa/build/main/pyproject.toml#project.description"
icon."$ref"       = "https://github.com/manifest.json#/icons/0"

At the time of writing, each of these results in the following

TOML

pypa_build = "A simple, correct Python build frontend"

[icon]
sizes = "114x114"
src = "https://github.githubassets.com/assets/apple-touch-icon-114x114-09ce42d3ca4b.png"

JSON

{
    "pypa_build": "A simple, correct Python build frontend",
    "icon": {
        "sizes": "114x114",
        "src": "https://github.githubassets.com/assets/apple-touch-icon-114x114-09ce42d3ca4b.png"
    }
}

Patches

An object or table with a $ref key can contain additional key/value pairs for patching. In this case, the reference value is copied.

The patch operations used here are defined in RFC 6902: JSON Patch using op, path, value and from fields. Path and from fields use the pointer representation of the document being patched.

Operation assign has been added as a simplified replace, dropping the requirement that the target location must exist for the operation to be successful. It also allows appending to an array with key -.

Operation merge has been added to merge an object into an object or extend an array with an array. When merging objects, existing keys will be replaced.

Three patch notations are allowed:

An array of operation objects

is a $patch key with array value, containing JSON Patch operation objects.

An array of shorthand operation arrays

is a $patch key with array value, containing shorthand operation arrays.

Key/value-pair assignments

are key/value pairs describing path/value assignments [3].

Shorthand operation arrays each consist of

• a shorthand op (+-@<$?=&),

• a path and

• a from or value field if relevant, depending on the operation.

Operation	Shorthand	Third element
add	+	value
remove	-	-
replace	@	value
move	<	from
copy	$	from
test	?	value
assign	=	value
merge	&	value

The $patch array can contain a mix of operation objects and shorthand operation arrays. $patch-array operations are applied in order of appearance, before any key/value-pair assignment.

Failing tests will raise an exception [4].

A patch is applied immediately after dereferencing the corresponding reference.

Example

TOML

[database]
ports        = [8000, 8001, 8002]
data         = [["delta", "phi"], [3.14]]
temp_targets = { cpu = 79.5, case = 72.0 }
enabled      = true

[rfc6902_patched]
"$ref" = "#database"
# RFC 6902 patch array
"$patch" = [
    { op = "assign", path = "data.0", value = "DeltaPhi" },
    { op = "replace", path = "data.1.0", value = 3.14159 },
    { op = "assign", path = "data.1.-", value = "radians" },
    { op = "add", path = "ports.-", value = 8003 },
    { op = "remove", path = "ports.0" },
    { op = "replace", path = "temp_targets.cpu", value = 80.0 },
    { op = "move", path = "target_temp_cpu", from = "temp_targets.cpu" },
    { op = "copy", path = "target_temp_case", from = "temp_targets.case" },
    { op = "test", path = "enabled", value = true },
    { op = "replace", path = "temp_targets.case", value = 75.0 },
    { op = "merge", path = "ports", value = [8004, 8005] },
    { op = "merge", path = "temp_targets", value = { lower = 7.0 } },
    { op = "add", path = "icon", value = { "$ref" = "https://api.github.com/emojis#/floppy_disk" } },
]

[shorthand_patched]
"$ref" = "#database"                                # replace shorthand_patched with reference
# key/value-pair assignment                         # Start patching:
"data.0" = "DeltaPhi"                               # - assign "DeltaPhi"
# $patch array
"$patch" = [
    ["@", "data.1.0", 3.14159],                     # - replace 3.14
    ["=", "data.1.-", "radians"],                   # - assign "radians" at array end
    ["+", "ports.-", 8003],                         # - add 8003 at end
    ["-", "ports.0"],                               # - remove 8000
    ["@", "temp_targets.cpu", 80.0],                # - replace 79.5 before move
    ["<", "target_temp_cpu", "temp_targets.cpu"],   # - move from temp_targets.cpu
    ["$", "target_temp_case", "temp_targets.case"], # - copy from temp_targets.case
    ["?", "enabled", true],                         # - test enabled
    # RFC 6902 notation                             # - replace 72.0 after copy
    { op = "replace", path = "temp_targets.case", value = 75.0 },
    ["&", "ports", [8004, 8005]],                   # - merge ports with [8004, 8005]
    ["&", "temp_targets", { lower = 7.0 }],         # - merge temp_targets with {...}
]
# key/value-pair assignment                         # - assign using reference
icon."$ref" = "https://api.github.com/emojis#/floppy_disk"

JSON

{
    "database": {
        "ports": [ 8000, 8001, 8002 ],
        "data": [ [ "delta", "phi" ], [ 3.14 ] ],
        "temp_targets": { "cpu": 79.5, "case": 72.0 },
        "enabled": true
    },
    "rfc6902_patched": {
        "$ref": "#/database",
        "$patch": [
            { "op": "assign", "path": "/data/0", "value": "DeltaPhi" },
            { "op": "replace", "path": "/data/1/0", "value": 3.14159 },
            { "op": "assign", "path": "/data/1/-", "value": "radians" },
            { "op": "add", "path": "/ports/-", "value": 8003 },
            { "op": "remove", "path": "/ports/0" },
            { "op": "replace", "path": "/temp_targets/cpu", "value": 80.0 },
            { "op": "move", "path": "/target_temp_cpu", "from": "/temp_targets/cpu" },
            { "op": "copy", "path": "/target_temp_case", "from": "/temp_targets/case" },
            { "op": "test", "path": "/enabled", "value": true },
            { "op": "replace", "path": "/temp_targets/case", "value": 75.0 },
            { "op": "merge", "path": "/ports", "value": [ 8004, 8005 ] },
            { "op": "merge", "path": "/temp_targets", "value": { "lower": 7.0 } },
            { "op": "add", "path": "/icon", "value": { "$ref": "https://api.github.com/emojis#/floppy_disk" } }
        ]
    },
    "shorthand_patched": {
        "$ref": "#/database",
        "/data/0": "DeltaPhi",
        "$patch": [
            [ "@", "/data/1/0", 3.14159 ],
            [ "=", "/data/1/-", "radians" ],
            [ "+", "/ports/-", 8003 ],
            [ "-", "/ports/0" ],
            [ "@", "/temp_targets/cpu", 80.0 ],
            [ "<", "/target_temp_cpu", "/temp_targets/cpu" ],
            [ "$", "/target_temp_case", "/temp_targets/case" ],
            [ "?", "/enabled", true ],
            { "op": "replace", "path": "/temp_targets/case", "value": 75.0 },
            [ "&", "/ports", [ 8004, 8005 ] ] ,
            [ "&", "/temp_targets", { "lower": 7.0 } ]
        ],
        "/icon": { "$ref": "https://api.github.com/emojis#/floppy_disk" }
    }
}

translates to

TOML

[database]
ports        = [8000, 8001, 8002]
data         = [["delta", "phi"], [3.14]]
temp_targets = { cpu = 79.5, case = 72.0 }
enabled      = true

[rfc6902_patched]
ports            = [8001, 8002, 8003, 8004, 8005]     # added 8003; removed 8000; merged with [8004, 8005]
data             = ["DeltaPhi", [3.14159, "radians"]] # assigned "DeltaPhi"; replaced 3.14; assigned "radians"
temp_targets     = { case = 75.0, lower = 7.0 }       # cpu moved away; replaced 72.0; merged with { lower = 7.0 }
enabled          = true                               # tested and passed
target_temp_cpu  = 80.0                               # moved from temp_targets, was replaced before move
target_temp_case = 72.0                               # copied from temp_targets, was replaced after copy
icon             = "https://github.githubassets.com/images/icons/emoji/unicode/1f4be.png?v8"

[shorthand_patched]
# Identical to [patched]
# ...

Full translation

TOML

[database]
ports = [
    8000,
    8001,
    8002,
]
data = [
    [
        "delta",
        "phi",
    ],
    [
        3.14,
    ],
]
enabled = true

[database.temp_targets]
cpu = 79.5
case = 72.0

[rfc6902_patched]
ports = [
    8001,
    8002,
    8003,
    8004,
    8005,
]
data = [
    "DeltaPhi",
    [
        3.14159,
        "radians",
    ],
]
enabled = true
target_temp_cpu = 80.0
target_temp_case = 72.0
icon = "https://github.githubassets.com/images/icons/emoji/unicode/1f4be.png?v8"

[rfc6902_patched.temp_targets]
case = 75.0
lower = 7.0

[shorthand_patched]
ports = [
    8001,
    8002,
    8003,
    8004,
    8005,
]
data = [
    "DeltaPhi",
    [
        3.14159,
        "radians",
    ],
]
enabled = true
target_temp_cpu = 80.0
target_temp_case = 72.0
icon = "https://github.githubassets.com/images/icons/emoji/unicode/1f4be.png?v8"

[shorthand_patched.temp_targets]
case = 75.0
lower = 7.0

JSON

{
    "database": {
        "ports": [
            8000,
            8001,
            8002
        ],
        "data": [
            [
                "delta",
                "phi"
            ],
            [
                3.14
            ]
        ],
        "temp_targets": {
            "cpu": 79.5,
            "case": 72.0
        },
        "enabled": true
    },
    "rfc6902_patched": {
        "ports": [
            8001,
            8002,
            8003,
            8004,
            8005
        ],
        "data": [
            "DeltaPhi",
            [
                3.14159,
                "radians"
            ]
        ],
        "temp_targets": {
            "case": 75.0,
            "lower": 7.0
        },
        "enabled": true,
        "target_temp_cpu": 80.0,
        "target_temp_case": 72.0,
        "icon": "https://github.githubassets.com/images/icons/emoji/unicode/1f4be.png?v8"
    },
    "shorthand_patched": {
        "ports": [
            8001,
            8002,
            8003,
            8004,
            8005
        ],
        "data": [
            "DeltaPhi",
            [
                3.14159,
                "radians"
            ]
        ],
        "temp_targets": {
            "case": 75.0,
            "lower": 7.0
        },
        "enabled": true,
        "target_temp_cpu": 80.0,
        "target_temp_case": 72.0,
        "icon": "https://github.githubassets.com/images/icons/emoji/unicode/1f4be.png?v8"
    }
}

Remarks

Definition implications

• Circular references are allowed, but references cannot point to themselves.

• Because references are resolved, and patches applied, depth-first, a $ref value can never be patched.

• Key/value-pair assignments may be applied out of order [5], so they shouldn’t be relied upon if patch order is of importance.

• Key/value-pair assignment allows only one replacement per path [6].

• A copy.deepcopy() is applied for reference substitution with patches.

[1]

Zero-based base-10 integers that give access to the corresponding array elements, or the single character “-” referencing the value past the last array element. The latter can be used to extend arrays.

[2]

URL fragments follow URL encoding.

[3]

Note that a key/value-pair assignment is impossible for a TOML key $patch. This will have to be part of a $patch array instead.

[4]

In this implementation, a rconf.patch.PatchTestError is raised for failing tests.

[5]

JSON RFC 8259 section 1 states

An object is an unordered collection of zero or more name/value pairs, …

TOML v1.0.0: Table mentions

Key/value pairs within tables are not guaranteed to be in any specific order.

[6]

JSON RFC 8259 section 4 states

The names within an object SHOULD be unique.

TOML v1.0.0: Keys mentions

Defining a key multiple times is invalid.