Skip to content

Reference

confarg

A tool to manage complex configurations.

Load and resolve complex configurations from files, environment variables and command line arguments. Keep your data structures and favorite CLI library.

ListOp

Bases: Enum

A list operation that merge() leaves pending in the dict it returns, for build() to apply.

A dict merge() returns may hold one of these as a key, beside or instead of the list it patches: an append with no base list below it yet, or an index patch that needs the field's type to apply. No string equals one, so a key a file, a variable or a flag spells the same way is always data. str() gives the reserved key a configuration file writes the op with, which is how dump_file() writes it and how merge(files=[...]) reads it back.

Example::

raw = confarg.merge(Config, argv=["--tags+", "x"])
# {'tags': {ListOp.APPEND: ['x']}}
str(confarg.ListOp.APPEND)  # '__append__'
Dev Notes

docs-dev/architecture/pipeline/deep-merge.md#no-channel-types-an-op

MergeOptions

Bases: TypedDict

The sources :func:confarg.merge and :func:confarg.load read, and how they read them.

Every key is optional and passed as a keyword argument::

confarg.load(MyConfig, env_prefix="MYAPP_", files=["defaults.yaml"])

The CLI adapters' merge_* and from_* functions take the same keywords. To forward them from a function of your own, annotate its keywords with this type::

def run(**opts: Unpack[confarg.MergeOptions]) -> MyConfig:
    return confarg.load(MyConfig, **opts)
Dev Notes

docs-dev/architecture/pipeline/api-seams.md#one-option-set

argv instance-attribute

argv

CLI arguments to parse. Defaults to sys.argv[1:].

A CLI adapter reads its CLI values and --config files from argv, so pass the list the framework parsed whenever it was not sys.argv[1:] (in tests, typically); a parse result holding a confarg flag that argv does not spell raises :class:~confarg.exceptions.ConfargError.

cli_prefix instance-attribute

cli_prefix

Namespace that all CLI flags live under, addressed as --<prefix>.<field>.

With one set, a flag that lacks it is rejected. It is also the only way to address a non-struct (scalar) target from the CLI, as --<prefix> VALUE.

Defaults to the prefix the flags were registered under: none for :func:confarg.merge and :func:confarg.load, and the one passed to populate_* for a CLI adapter, which is where an adapter applies it. Passing an adapter a prefix that disagrees with the registered one raises :class:~confarg.exceptions.ConfargError rather than silently matching no flags.

config_flag instance-attribute

config_flag

Name of the CLI flag that loads config files, --config path/to/file.yaml.

It also names the environment variable that does, <env_prefix>CONFIG=path/to/file.yaml. Defaults to "config"; "" disables both. A CLI adapter must be given the config_flag passed to populate_*.

env instance-attribute

env

Environment variable mapping to scan. Defaults to :data:os.environ.

env_config instance-attribute

env_config

Name of an environment variable whose value is a config file path to load.

The file is loaded after files but before CLI --config files.

env_prefix instance-attribute

env_prefix

Prefix that environment variables must start with to be read.

Defaults to None, which disables environment variable parsing entirely. Set to "" to read every variable, or to e.g. "MYAPP_" to read only the variables with that prefix.

env_separator instance-attribute

env_separator

Separator splitting environment variable names into nested keys.

Defaults to "__": MYAPP_DB__MAX_CONNECTIONS sets db.max_connections.

files instance-attribute

files

Config files to load first, at the lowest priority, in the order given.

union_tag instance-attribute

union_tag

Field name used as a discriminator tag in union types. Defaults to "class".

build

build(
    target: type[T],
    data: dict[Any, Any],
    *,
    union_tag: str = ...,
) -> T
build(
    target: object,
    data: dict[Any, Any],
    *,
    union_tag: str = ...,
) -> Any
build(target, data, *, union_tag=_defaults.UNION_TAG)

Resolve ${...} expressions and construct the target type from a merged config dict.

Use this as the second step after merge(), or to load configuration from a dict you have assembled yourself.

Parameters:

Name Type Description Default
target type[T] | TypeAliasType | UnionType

The dataclass type (or scalar type) to construct.

required
data dict[Any, Any]

The raw config dict (e.g. the output of merge()).

required
union_tag str

The field name used as a discriminator tag in unions.

UNION_TAG

Returns:

Type Description
T

An instance of the target type.

Raises:

Type Description
MissingFieldError

If a required field is not provided.

TypeCoercionError

If a value cannot be coerced to the target type.

AmbiguousUnionError

If a Union cannot be disambiguated.

CircularReferenceError

If expression references form a cycle.

UnsafeExpressionError

If an expression contains disallowed constructs.

MissingReferenceError

If an expression references a field that does not exist.

ExpressionEvalError

If an expression fails at runtime.

dump

dump(
    value,
    *,
    union_tag=_defaults.UNION_TAG,
    tag_policy="auto",
)

Serialize a dataclass instance to a config-compatible plain dict.

The dump records the values the instance holds, not how they were produced: ${...} expressions are written as their resolved values, and a value a call: spec computed is written as that value, not as the call. To save a configuration as written, with its expressions and its call: specs, pass the dict returned by merge() to dump_file() instead. Plain (non-dataclass) classes are not supported.

Parameters:

Name Type Description Default
value Any

A dataclass instance.

required
union_tag str

The field name used as a discriminator tag in unions.

UNION_TAG
tag_policy TagPolicy

"auto" (tag only when needed) or "always" (tag every union member).

'auto'

Returns:

Type Description
dict[str, Any]

A plain dict representation.

Raises:

Type Description
TypeError

If value is not a dataclass instance.

dump_file

dump_file(
    value,
    path,
    *,
    union_tag=_defaults.UNION_TAG,
    tag_policy="auto",
)

Serialize and write to a config file.

Accepts a dataclass instance or a raw config dict (e.g. from merge() or resolve()). A dict is written as it stands, so the dict merge() returns keeps its ${...} expressions and call: specs; an instance is written as dump() serializes it, as the values it holds. The output format is determined by the file extension (.toml, .yaml, .yml, .json).

Parameters:

Name Type Description Default
value Any

A dataclass instance or a raw config dict.

required
path str | Path

Path to the output file.

required
union_tag str

The field name used as a discriminator tag in unions.

UNION_TAG
tag_policy TagPolicy

"auto" or "always".

'auto'

Raises:

Type Description
TypeError

If value is not a dataclass instance or a dict.

InvalidConfigFileError

If the format is unsupported or the required library is not installed.

from_dict

from_dict(
    target: type[T],
    data: dict[Any, Any],
    *,
    union_tag: str = ...,
) -> T
from_dict(
    target: object,
    data: dict[Any, Any],
    *,
    union_tag: str = ...,
) -> Any
from_dict(target, data, *, union_tag=_defaults.UNION_TAG)

Construct a typed object from an already-resolved config dict.

Unlike build(), this does NOT resolve ${...} expressions — call resolve() first if needed.

Parameters:

Name Type Description Default
target type[T] | TypeAliasType | UnionType

The dataclass or plain-class type to construct.

required
data dict[Any, Any]

A resolved config dict (output of resolve() or merge()).

required
union_tag str

The field name used as a discriminator tag in unions.

UNION_TAG

Returns:

Type Description
T

An instance of the target type.

Raises:

Type Description
MissingFieldError

If a required field is not provided.

TypeCoercionError

If a value cannot be coerced to the target type.

AmbiguousUnionError

If a Union cannot be disambiguated.

load

load(target: type[T], **opts: Unpack[MergeOptions]) -> T
load(target: object, **opts: Unpack[MergeOptions]) -> Any
load(target, **opts)

Merge configuration from all sources and construct the target type.

Convenience wrapper for merge() + build(). For more control — e.g. to inspect or save the raw merged dict — call those directly. See merge() for source priority and config file loading order.

Parameters:

Name Type Description Default
target type[T] | TypeAliasType | UnionType

The dataclass type (or scalar type) to load configuration into.

required
**opts Unpack[MergeOptions]

The sources to read and how to read them; see :class:MergeOptions.

{}

Returns:

Type Description
T

An instance of the target type populated with the merged configuration.

Raises:

Type Description
MissingFieldError

If a required field is not provided by any source.

TypeCoercionError

If a value cannot be coerced to the target type.

InvalidConfigFileError

If a config file cannot be loaded.

UnknownArgumentError

If an unrecognized CLI argument is encountered.

AmbiguousUnionError

If a Union cannot be disambiguated.

CircularReferenceError

If expression references form a cycle.

UnsafeExpressionError

If an expression contains disallowed constructs.

MissingReferenceError

If an expression references a field that does not exist.

ExpressionEvalError

If an expression fails at runtime.

merge

merge(target, **opts)

Collect and merge configuration from all sources into a raw dict.

Sources are merged in priority order: config files (lowest), then environment variables, then CLI arguments (highest). No expression resolution and no dataclass construction are performed — the returned dict reflects the config input exactly as written, with ${...} expression strings preserved.

The returned dict is unvalidated: it is not guaranteed to be constructible into target. Type validation and coercion happen in build() (or the one-shot load()), which raises TypeCoercionError / MissingFieldError for data that cannot be built. CLI parsing rejects only the cases it can prove wrong at parse time (e.g. a missing value); a config file or env var carrying "abc" for an int field merges cleanly here and fails later, in build().

Parameters:

Name Type Description Default
target type | TypeAliasType | UnionType

The dataclass type (or scalar type) used to guide CLI parsing.

required
**opts Unpack[MergeOptions]

The sources to read and how to read them; see :class:MergeOptions.

{}

Returns:

Type Description
dict[str | ListOp, Any]

A plain dict of the merged configuration, with expression strings intact. A list

dict[str | ListOp, Any]

operation that only build() can apply stays in it, keyed by a

dict[str | ListOp, Any]

class:confarg.ListOp.

Config file loading order

All config files share the same priority level (below inline env vars and CLI args). Within that level they are loaded left-to-right so that later sources win on conflict. The full sequence is:

  1. files — in the order given.
  2. env_config — the single global path named by that env var (if set).
  3. CONFIG__* env vars — sorted lexicographically by their env var name, which is equivalent to sorting by subpath depth (shallower paths first). A global CONFIG=file therefore loads before CONFIG__DB=db.yaml, which loads before CONFIG__DB__HOST=host.yaml.
  4. CLI --config / --config.subpath flags — in left-to-right order.

Raises:

Type Description
InvalidConfigFileError

If a config file cannot be loaded.

UnknownArgumentError

If an unrecognized CLI argument is encountered.

register_leaf_type

register_leaf_type(tp, coerce, *, serialize=str)

Register a custom leaf type.

After registration, tp is treated as a leaf in both directions: values of that type can appear as scalars in config files, env vars, and CLI args, and they are written back as scalars by dump() and dump_file(), even if tp would otherwise be constructed field by field as a class.

coerce is called with the raw input value — either a str subclass from CLI, env and CSV sources, or the natively-parsed Python object from config files — and must return an instance of tp, raising ValueError, TypeError or OSError on failure. Values that already are instances of tp are used as-is. Unresolved ${...} expressions are never passed to coerce; it receives their resolved value.

serialize is the inverse: it is called with an instance of tp and must return a value a config file can hold (a string, number or bool), which coerce accepts back. It defaults to str, which is right for types whose str() is their wire form (UUID, Decimal, Path) and wrong for types whose __str__ is a repr-style debugging aid — pass an explicit callable for those, or dumps will not load back.

Example::

from uuid import UUID
confarg.register_leaf_type(UUID, UUID)            # str() is the wire form

confarg.register_leaf_type(Int, coerce_int, serialize=lambda i: i.value)

register_scheme

register_scheme(scheme, read)

Register a handler that reads config sources for a URL scheme.

After registration, any location beginning <scheme>: can be used wherever a configuration file path can: files=, env_config, <PREFIX>CONFIG__<SUBPATH>, --config[.subpath][+] and __include__.

read is called with the full location string exactly as it was written -- "gs://bucket/path/config.yaml" -- and must return the document's raw bytes. confarg decodes them itself, so a handler never chooses an encoding: that keeps a BOM working in JSON and CSV, which a decoded str would break. Raise OSError (or confarg.exceptions.InvalidConfigFileError) when the document cannot be read; anything else propagates unchanged.

The format is still chosen by the extension of the location's path, so a handler is not asked about it and a location with no extension is rejected. Relative __include__ paths inside a document fetched this way resolve against its own location and may not leave its scheme and host.

file, http, https and s3 are registered already (s3 needs boto3: pip install confarg[s3]). Registering one of those names again replaces it, which is how an application adds authentication, a different timeout, or caching.

Example::

import urllib.request

def read_authenticated(location: str) -> bytes:
    request = urllib.request.Request(location, headers={"Authorization": TOKEN})
    with urllib.request.urlopen(request, timeout=30) as response:
        return response.read()

confarg.register_scheme("https", read_authenticated)   # replace the built-in
confarg.register_scheme("gs", read_gcs)                # or add a new scheme

resolve

resolve(target, data, *, union_tag=_defaults.UNION_TAG)

Resolve ${...} expressions in a merged config dict, over the target's field defaults.

Use this between merge() and from_dict() when you need the resolved dict itself (e.g. to inspect values or write it to a file):

raw = confarg.merge(MyConfig, ...)
resolved = confarg.resolve(MyConfig, raw)
confarg.dump_file(resolved, "out.yaml")
cfg = confarg.from_dict(MyConfig, resolved)

The target's field defaults are the layer below data: an expression reads a default nobody set, and a patch the merge left pending over a defaulted field (--t+ 3) is applied over the default. A default nothing touched is not in the result; construction applies it.

Parameters:

Name Type Description Default
target type | TypeAliasType | UnionType

The type data is for, whose field defaults lie below it; dict for none.

required
data dict[Any, Any]

A plain config dict, e.g. the output of merge().

required
union_tag str

The field name used as a discriminator tag in unions.

UNION_TAG

Returns:

Type Description
dict[str | ListOp, Any]

A new dict with all ${...} expression strings replaced by their values.

Raises:

Type Description
ConfargError

If a patch the merge left pending does not apply over a default.

CircularReferenceError

If expression references form a cycle.

UnsafeExpressionError

If an expression contains disallowed constructs.

MissingReferenceError

If an expression references a field that does not exist.

ExpressionEvalError

If an expression fails at runtime.

Dev Notes

docs-dev/architecture/design-decisions/a-field-default-is-the-base-layer.md#a-field-default-is-the-base-layer

unregister_scheme

unregister_scheme(scheme)

Remove the reader registered for a URL scheme, a built-in one included.

A location naming the scheme afterwards is refused the way any unknown scheme is, with InvalidConfigFileError naming the schemes that remain -- from every channel, since the registry is global. unregister_scheme("http") and unregister_scheme("https") are therefore how an application forbids loading configuration over the network at all::

confarg.unregister_scheme("http")
confarg.unregister_scheme("https")
confarg.load(Config, files=["https://cfg.example.com/app.yaml"])  # raises

The scheme name is matched case-insensitively and may be written with its separator ("HTTP", "http:" and "http://" all name the same entry).

Removing a scheme is not how a handler is replaced -- pass the new reader to :func:register_scheme instead, which overwrites in place.

Raises:

Type Description
ValueError

If no reader is registered for the scheme. Removing what is not there is a mistake worth hearing about rather than a silent no-op, because the usual cause is a misspelled scheme name and the silent version leaves the scheme loadable.