quant_init

fastforward.quantization.quant_init.QuantizationConfig()

Manages quantization configuration rules and initializes quantizers in a model.

__repr__()

add_rule(query, factory, **kwargs)

Add a quantizer rule to the collection.

A rule consists of an mpath query and a quantizer factory. When a query matches a quantizer (stub) and has highest precedence, the factory is called to initialize a Quantizer (see the docstring of precedence_score).

quantizer_factory may either be a Quantizer subclass or a callable that accepts a str and Quantizer. In the latter case, the provided string is the full name relative to the root module and the quantizer is the quantizer that is currently referenced at this location. This may either be a QuantizerStub or an existing (and initialized) quantizer.

Parameters:

Name	Type	Description	Default
query	BaseSelector | str	Mpath.Selector or str that represent the filter query. Please see the documentation of fastforward.mpath.query and fastforward.mpath.search for more details.	required
quantizer_factory		Quantizer or callable to initialize each quantizer in the collection. This factory is called for each element in the collection. If a Quantizer class is passed, all kwargs will be forwarded to the initializer. In the case of a callable, all kwargs are ignored.	required

initialize(model, *, safe=True, overwrite_policy='error')

Initialize quantizers in model following the rules in this config.

For each quantizer (stub) that matches at least one location, the last added rule is applied. The factory function of this rule is executed to obtain an initialized quantizer.

Parameters:

Name	Type	Description	Default
model	Module	Root module to initialize quantizers in	required
overwrite_policy	_OverwriteOptions	Either "skip", "overwrite", or "error". The overwrite_policy indicates what to do when the element in the collection is already an initialized quantizer. In case of "skip" the initialization for that specific quantizer is skipped and nothing happens, for "overwrite" a new quantizer is created and the existing quantizer is overwritten. Lastly, in case of "error" an error is raised when it is attempted to re-initialize an already initialized quantizer.	'error'
safe	bool	Boolean that indicates what to do when the module was already replaced in the root module between the creation of this collection and the call to initialize. When safe is True, an error is raised. Otherwise, a (re-)initialization of the quantizer is attempted.	True

precedence_score(rule, result)

Score precedence given a result.

Returns an integer value that is used to choose between two matching rules. The rule with the highest score is selected. Ties are broken by order, i.e., select the rule that was added last. The precedence score is computer on a per result basis, this means that the precedence can differ per module.

Parameters:

Name	Type	Description	Default
rule	_QuantConfigRule	The rule that is scored	required
result	FilterResult	The query/filter request for a given module obtained using rule	required

Returns:

Type	Description
int	integer indicating the rule/result precedence

fastforward.quantization.quant_init.QuantizerCollection(root, results=None)

Bases: MPathCollection

MPathCollection that only stores Quantizer subclasses.

On creation, all non quantizer results are filtered out from the results iterable. In contrast, append will throw an error when a non quantizer result is provided.

Parameters:

Name	Type	Description	Default
root	Module	The root module that was used to produce the search results	required
results	Optional[Iterable[FilterResult]]	Optional list of search results, may be extended using append	None

append(item)

Add FilterResult to the collection.

Raises an error if the filter result contains a module that is not a Quantizer.

Parameters:

Name	Type	Description	Default
item	FilterResult	Filter result to add to the collection	required

initialize(quantizer_factory, *, overwrite_policy='error', safe=True, **kwargs)

Initialize all quantizers in the collection using quantizer_factory.

quantizer_factory may either be a Quantizer subclass or a callable that accepts a str and Quantizer. In the latter case, the provided string is the full name relative to the root module and the quantizer is the quantizer that is currently referenced at this location. This may either be a QuantizerStub or an existing quantizer.

Parameters:

Name	Type	Description	Default
quantizer_factory	type[Quantizer] | Callable[[str, Module], Quantizer]	Quantizer or callable to initialize each quantizer in the collection. This factory is called for each element in the collection. If a Quantizer class is passed, all kwargs will be forwarded to the initializer. In the case of a callable, all kwargs are ignored.	required
overwrite_policy	_OverwriteOptions	Either "skip", "overwrite", or "error". The overwrite_policy indicates what to do when the element in the collection is already an initialized quantizer. In case of "skip" the initialization for that specific quantizer is skipped and nothing happens, for "overwrite" a new quantizer is created and the existing quantizer is overwritten. Lastly, in case of "error" an error is raised when it is attempted to re-initialize an already initialized quantizer.	'error'
safe	bool	Boolean that indicates what to do when the module was already replaced in the root module between the creation of this collection and the call to initialize. When safe is True, an error is raised. Otherwise, a (re-)initialization of the quantizer is attempted.	True

fastforward.quantization.quant_init.QuantizerTagSelectorFragment(*tags)

Bases: Fragment

Fragment that matches quantizer tags.

Can be used in query string using [quantizer:<tag>(,<tag>)+] or "[qtag:(,)+]

__hash__()

__str__()

from_raw_string(raw_str) classmethod

Create a QuantizerTagSelectorFragment from a string.

Tags are expected to be comma separated. Any whitespace between tags is ignored.

match(fragment_name, module)

Return True if module's tag metadata includes self._tags, False otherwise.

Parameters:

Name	Type	Description	Default
fragment_name	str	Unused	required
module	Module	The module that corresponds to the fragment name.	required

Returns: Boolean indicating whether the fragment matches the tag set.

fastforward.quantization.quant_init.find_quantizers(root_module, query, *, aliases=None)

Find all quantizers in root_module that match query.

Parameters:

Name	Type	Description	Default
root_module	Module	The module to search for quantizers.	required
query	str | BaseSelector	Mpath.Selector or str that represent the filter query. Please see the documentation of fastforward.mpath.query and fastforward.mpath.search for more details.	required