Reference
confarg
¶
A tool to manage complex configurations.
Load and resolve complex configurations from files, environment variables and command line arguments. Keep your data structures and favorite CLI library.
ListOp
¶
Bases: Enum
A list operation that merge() leaves pending in the dict it returns, for build() to apply.
A dict merge() returns may hold one of these as a key, beside or instead of the list it
patches: an append with no base list below it yet, or an index patch that needs the
field's type to apply. No string equals one, so a key a file, a variable or a flag spells
the same way is always data. str() gives the reserved key a configuration file writes
the op with, which is how dump_file() writes it and how merge(files=[...]) reads it back.
Example::
raw = confarg.merge(Config, argv=["--tags+", "x"])
# {'tags': {ListOp.APPEND: ['x']}}
str(confarg.ListOp.APPEND) # '__append__'
Dev Notes
docs-dev/architecture/pipeline/deep-merge.md#no-channel-types-an-op
MergeOptions
¶
Bases: TypedDict
The sources :func:confarg.merge and :func:confarg.load read, and how they read them.
Every key is optional and passed as a keyword argument::
confarg.load(MyConfig, env_prefix="MYAPP_", files=["defaults.yaml"])
The CLI adapters' merge_* and from_* functions take the same keywords. To
forward them from a function of your own, annotate its keywords with this type::
def run(**opts: Unpack[confarg.MergeOptions]) -> MyConfig:
return confarg.load(MyConfig, **opts)
Dev Notes
docs-dev/architecture/pipeline/api-seams.md#one-option-set
argv
instance-attribute
¶
CLI arguments to parse. Defaults to sys.argv[1:].
A CLI adapter reads its CLI values and --config files from argv, so pass the list
the framework parsed whenever it was not sys.argv[1:] (in tests, typically); a parse
result holding a confarg flag that argv does not spell raises
:class:~confarg.exceptions.ConfargError.
cli_prefix
instance-attribute
¶
Namespace that all CLI flags live under, addressed as --<prefix>.<field>.
With one set, a flag that lacks it is rejected. It is also the only way to address a
non-struct (scalar) target from the CLI, as --<prefix> VALUE.
Defaults to the prefix the flags were registered under: none for :func:confarg.merge
and :func:confarg.load, and the one passed to populate_* for a CLI adapter, which
is where an adapter applies it. Passing an adapter a prefix that disagrees with the
registered one raises :class:~confarg.exceptions.ConfargError rather than silently
matching no flags.
config_flag
instance-attribute
¶
Name of the CLI flag that loads config files, --config path/to/file.yaml.
It also names the environment variable that does, <env_prefix>CONFIG=path/to/file.yaml.
Defaults to "config"; "" disables both. A CLI adapter must be given the
config_flag passed to populate_*.
env_config
instance-attribute
¶
Name of an environment variable whose value is a config file path to load.
The file is loaded after files but before CLI --config files.
env_prefix
instance-attribute
¶
Prefix that environment variables must start with to be read.
Defaults to None, which disables environment variable parsing entirely. Set to
"" to read every variable, or to e.g. "MYAPP_" to read only the variables with
that prefix.
env_separator
instance-attribute
¶
Separator splitting environment variable names into nested keys.
Defaults to "__": MYAPP_DB__MAX_CONNECTIONS sets db.max_connections.
files
instance-attribute
¶
Config files to load first, at the lowest priority, in the order given.
union_tag
instance-attribute
¶
Field name used as a discriminator tag in union types. Defaults to "class".
build
¶
Resolve ${...} expressions and construct the target type from a merged config dict.
Use this as the second step after merge(), or to load configuration
from a dict you have assembled yourself.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
target
|
type[T] | TypeAliasType | UnionType
|
The dataclass type (or scalar type) to construct. |
required |
data
|
dict[Any, Any]
|
The raw config dict (e.g. the output of |
required |
union_tag
|
str
|
The field name used as a discriminator tag in unions. |
UNION_TAG
|
Returns:
| Type | Description |
|---|---|
T
|
An instance of the target type. |
Raises:
| Type | Description |
|---|---|
MissingFieldError
|
If a required field is not provided. |
TypeCoercionError
|
If a value cannot be coerced to the target type. |
AmbiguousUnionError
|
If a Union cannot be disambiguated. |
CircularReferenceError
|
If expression references form a cycle. |
UnsafeExpressionError
|
If an expression contains disallowed constructs. |
MissingReferenceError
|
If an expression references a field that does not exist. |
ExpressionEvalError
|
If an expression fails at runtime. |
dump
¶
Serialize a dataclass instance to a config-compatible plain dict.
The dump records the values the instance holds, not how they were produced:
${...} expressions are written as their resolved values, and a value a call:
spec computed is written as that value, not as the call. To save a configuration as
written, with its expressions and its call: specs, pass the dict returned by
merge() to dump_file() instead. Plain (non-dataclass) classes are not
supported.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
Any
|
A dataclass instance. |
required |
union_tag
|
str
|
The field name used as a discriminator tag in unions. |
UNION_TAG
|
tag_policy
|
TagPolicy
|
|
'auto'
|
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
A plain dict representation. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If value is not a dataclass instance. |
dump_file
¶
Serialize and write to a config file.
Accepts a dataclass instance or a raw config dict (e.g. from merge()
or resolve()). A dict is written as it stands, so the dict merge()
returns keeps its ${...} expressions and call: specs; an instance is
written as dump() serializes it, as the values it holds. The output
format is determined by the file extension (.toml, .yaml, .yml, .json).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
Any
|
A dataclass instance or a raw config dict. |
required |
path
|
str | Path
|
Path to the output file. |
required |
union_tag
|
str
|
The field name used as a discriminator tag in unions. |
UNION_TAG
|
tag_policy
|
TagPolicy
|
|
'auto'
|
Raises:
| Type | Description |
|---|---|
TypeError
|
If value is not a dataclass instance or a dict. |
InvalidConfigFileError
|
If the format is unsupported or the required library is not installed. |
from_dict
¶
Construct a typed object from an already-resolved config dict.
Unlike build(), this does NOT resolve ${...} expressions — call
resolve() first if needed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
target
|
type[T] | TypeAliasType | UnionType
|
The dataclass or plain-class type to construct. |
required |
data
|
dict[Any, Any]
|
A resolved config dict (output of resolve() or merge()). |
required |
union_tag
|
str
|
The field name used as a discriminator tag in unions. |
UNION_TAG
|
Returns:
| Type | Description |
|---|---|
T
|
An instance of the target type. |
Raises:
| Type | Description |
|---|---|
MissingFieldError
|
If a required field is not provided. |
TypeCoercionError
|
If a value cannot be coerced to the target type. |
AmbiguousUnionError
|
If a Union cannot be disambiguated. |
load
¶
Merge configuration from all sources and construct the target type.
Convenience wrapper for merge() + build(). For more control —
e.g. to inspect or save the raw merged dict — call those directly.
See merge() for source priority and config file loading order.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
target
|
type[T] | TypeAliasType | UnionType
|
The dataclass type (or scalar type) to load configuration into. |
required |
**opts
|
Unpack[MergeOptions]
|
The sources to read and how to read them; see :class: |
{}
|
Returns:
| Type | Description |
|---|---|
T
|
An instance of the target type populated with the merged configuration. |
Raises:
| Type | Description |
|---|---|
MissingFieldError
|
If a required field is not provided by any source. |
TypeCoercionError
|
If a value cannot be coerced to the target type. |
InvalidConfigFileError
|
If a config file cannot be loaded. |
UnknownArgumentError
|
If an unrecognized CLI argument is encountered. |
AmbiguousUnionError
|
If a Union cannot be disambiguated. |
CircularReferenceError
|
If expression references form a cycle. |
UnsafeExpressionError
|
If an expression contains disallowed constructs. |
MissingReferenceError
|
If an expression references a field that does not exist. |
ExpressionEvalError
|
If an expression fails at runtime. |
merge
¶
Collect and merge configuration from all sources into a raw dict.
Sources are merged in priority order: config files (lowest), then environment variables, then CLI arguments (highest). No expression resolution and no dataclass construction are performed — the returned dict reflects the config input exactly as written, with ${...} expression strings preserved.
The returned dict is unvalidated: it is not guaranteed to be
constructible into target. Type validation and coercion happen in
build() (or the one-shot load()), which raises TypeCoercionError
/ MissingFieldError for data that cannot be built. CLI parsing rejects
only the cases it can prove wrong at parse time (e.g. a missing value); a
config file or env var carrying "abc" for an int field merges
cleanly here and fails later, in build().
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
target
|
type | TypeAliasType | UnionType
|
The dataclass type (or scalar type) used to guide CLI parsing. |
required |
**opts
|
Unpack[MergeOptions]
|
The sources to read and how to read them; see :class: |
{}
|
Returns:
| Type | Description |
|---|---|
dict[str | ListOp, Any]
|
A plain dict of the merged configuration, with expression strings intact. A list |
dict[str | ListOp, Any]
|
operation that only |
dict[str | ListOp, Any]
|
class: |
Config file loading order
All config files share the same priority level (below inline env vars and CLI args). Within that level they are loaded left-to-right so that later sources win on conflict. The full sequence is:
files— in the order given.env_config— the single global path named by that env var (if set).CONFIG__*env vars — sorted lexicographically by their env var name, which is equivalent to sorting by subpath depth (shallower paths first). A globalCONFIG=filetherefore loads beforeCONFIG__DB=db.yaml, which loads beforeCONFIG__DB__HOST=host.yaml.- CLI
--config/--config.subpathflags — in left-to-right order.
Raises:
| Type | Description |
|---|---|
InvalidConfigFileError
|
If a config file cannot be loaded. |
UnknownArgumentError
|
If an unrecognized CLI argument is encountered. |
register_leaf_type
¶
Register a custom leaf type.
After registration, tp is treated as a leaf in both directions: values of
that type can appear as scalars in config files, env vars, and CLI args, and
they are written back as scalars by dump() and dump_file(), even if
tp would otherwise be constructed field by field as a class.
coerce is called with the raw input value — either a str subclass
from CLI, env and CSV sources, or the natively-parsed Python object from
config files — and must return an instance of tp, raising ValueError,
TypeError or OSError on failure. Values that already are instances of
tp are used as-is. Unresolved ${...} expressions are never passed to
coerce; it receives their resolved value.
serialize is the inverse: it is called with an instance of tp and must
return a value a config file can hold (a string, number or bool), which
coerce accepts back. It defaults to str, which is right for types
whose str() is their wire form (UUID, Decimal, Path) and wrong
for types whose __str__ is a repr-style debugging aid — pass an
explicit callable for those, or dumps will not load back.
Example::
from uuid import UUID
confarg.register_leaf_type(UUID, UUID) # str() is the wire form
confarg.register_leaf_type(Int, coerce_int, serialize=lambda i: i.value)
register_scheme
¶
Register a handler that reads config sources for a URL scheme.
After registration, any location beginning <scheme>: can be used wherever a
configuration file path can: files=, env_config, <PREFIX>CONFIG__<SUBPATH>,
--config[.subpath][+] and __include__.
read is called with the full location string exactly as it was written --
"gs://bucket/path/config.yaml" -- and must return the document's raw bytes.
confarg decodes them itself, so a handler never chooses an encoding: that keeps a BOM
working in JSON and CSV, which a decoded str would break. Raise OSError (or
confarg.exceptions.InvalidConfigFileError) when the document cannot be read; anything
else propagates unchanged.
The format is still chosen by the extension of the location's path, so a handler is not
asked about it and a location with no extension is rejected. Relative __include__
paths inside a document fetched this way resolve against its own location and may not
leave its scheme and host.
file, http, https and s3 are registered already (s3 needs boto3:
pip install confarg[s3]). Registering one of those names again replaces it, which is
how an application adds authentication, a different timeout, or caching.
Example::
import urllib.request
def read_authenticated(location: str) -> bytes:
request = urllib.request.Request(location, headers={"Authorization": TOKEN})
with urllib.request.urlopen(request, timeout=30) as response:
return response.read()
confarg.register_scheme("https", read_authenticated) # replace the built-in
confarg.register_scheme("gs", read_gcs) # or add a new scheme
resolve
¶
Resolve ${...} expressions in a merged config dict, over the target's field defaults.
Use this between merge() and from_dict() when you need the
resolved dict itself (e.g. to inspect values or write it to a file):
raw = confarg.merge(MyConfig, ...)
resolved = confarg.resolve(MyConfig, raw)
confarg.dump_file(resolved, "out.yaml")
cfg = confarg.from_dict(MyConfig, resolved)
The target's field defaults are the layer below data: an expression reads a default
nobody set, and a patch the merge left pending over a defaulted field (--t+ 3) is
applied over the default. A default nothing touched is not in the result; construction
applies it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
target
|
type | TypeAliasType | UnionType
|
The type data is for, whose field defaults lie below it; |
required |
data
|
dict[Any, Any]
|
A plain config dict, e.g. the output of |
required |
union_tag
|
str
|
The field name used as a discriminator tag in unions. |
UNION_TAG
|
Returns:
| Type | Description |
|---|---|
dict[str | ListOp, Any]
|
A new dict with all |
Raises:
| Type | Description |
|---|---|
ConfargError
|
If a patch the merge left pending does not apply over a default. |
CircularReferenceError
|
If expression references form a cycle. |
UnsafeExpressionError
|
If an expression contains disallowed constructs. |
MissingReferenceError
|
If an expression references a field that does not exist. |
ExpressionEvalError
|
If an expression fails at runtime. |
Dev Notes
docs-dev/architecture/design-decisions/a-field-default-is-the-base-layer.md#a-field-default-is-the-base-layer
unregister_scheme
¶
Remove the reader registered for a URL scheme, a built-in one included.
A location naming the scheme afterwards is refused the way any unknown scheme is, with
InvalidConfigFileError naming the schemes that remain -- from every channel, since the
registry is global. unregister_scheme("http") and unregister_scheme("https") are
therefore how an application forbids loading configuration over the network at all::
confarg.unregister_scheme("http")
confarg.unregister_scheme("https")
confarg.load(Config, files=["https://cfg.example.com/app.yaml"]) # raises
The scheme name is matched case-insensitively and may be written with its separator
("HTTP", "http:" and "http://" all name the same entry).
Removing a scheme is not how a handler is replaced -- pass the new reader to
:func:register_scheme instead, which overwrites in place.
Raises:
| Type | Description |
|---|---|
ValueError
|
If no reader is registered for the scheme. Removing what is not there is a mistake worth hearing about rather than a silent no-op, because the usual cause is a misspelled scheme name and the silent version leaves the scheme loadable. |