Input layer — Schema & from_dataframe

Summary

effector is numpy-only at the border (design contract R10): data is a 2-D numeric numpy array, the model is a numpy→numpy callable, and all input metadata travels in the single schema= argument — an effector.Schema (frozen dataclass, reusable across constructions) or a plain dict with the same keys. Every field is optional; anything omitted is inferred conservatively (nominal is never guessed from a numpy matrix).

schema = {
    "feature_names": ["hr", "temp", "workingday"],
    "feature_types": ["ordinal", "continuous", "nominal"],
    "target_name": "count",
}
pdp = effector.PDP(X, model, schema=schema)

Coming from pandas, from_dataframe reads names, dtypes, and category levels into (X, Schema) — it converts data only and never touches the model. The returned schema is a proposal to inspect: the int-column guess (ordinal vs continuous vs label-encoded nominal) is the one thing no extractor can know for sure.

X, schema = effector.from_dataframe(df)
ale = effector.ALE(X, model, schema=schema)

(For the model side of the border — sklearn/torch/classifier wrappers — see Adapters.)

---

API

effector.ingestion.Schema(feature_names=None, feature_types=None, cat_limit=None, target_name=None, scale_x_list=None, scale_y=None, category_names=None) dataclass

The single metadata argument of every effector constructor.

schema = effector.Schema(
    feature_names=["hr", "temp", "workingday"],
    feature_types=["ordinal", "continuous", "nominal"],
    target_name="count",
)
pdp = effector.PDP(X, model, schema=schema)

Every field is optional: whatever you do not declare is inferred from the numpy data (type heuristic) or synthesized (x_0…, "y"). A Schema holds no data, so one instance can be reused across method constructions. Constructors also accept a plain dict with the same keys.

Field	Meaning
feature_names	one name per column (default x_0, x_1, …)
feature_types	"continuous" / "ordinal" / "nominal" per column (aliases: "cont" → continuous, "cat" → nominal)
cat_limit	cardinality threshold for the int-column type heuristic (default 10)
target_name	name of the model output (default "y")
scale_x_list	per-feature {"mean": .., "std": ..} dicts (or None entries) to display plots in original units; plot-time scale_x overrides
scale_y	{"mean": .., "std": ..} for the output axis; plot-time scale_y overrides
category_names	per categorical (ordinal/nominal) feature, the human-readable level names shown on the plot axis instead of the numeric codes: a {level value: name} dict (levels absent from the data are fine), or a list with one name per observed level in ascending order; None entries keep the codes

Coming from pandas

effector.from_dataframe(df) populates a Schema from a DataFrame's names/dtypes/levels — inspect the result, then pass it on.

effector.ingestion.from_dataframe(df, *, cat_limit=DEFAULT_CAT_LIMIT, return_encoding=False)

Extract a numpy matrix and a populated Schema from a pandas DataFrame — data only.

X, schema = effector.from_dataframe(df)
pdp = effector.PDP(X, model, schema=schema)

Constructors hard-reject a DataFrame as data (effector is numpy-only); this is the opt-in door. It reads column names, maps dtypes to feature types, encodes non-numeric columns to float codes, and records the level labels in category_names (a {code: name} dict per encoded column):

DataFrame dtype	feature type	encoded as
float	continuous	values as-is
int	ordinal if nunique < cat_limit, else continuous	values as-is
bool	ordinal	False, True → 0.0, 1.0
Categorical, ordered	ordinal	category codes
Categorical, unordered	nominal	category codes
object / string	nominal	codes of astype("category")
anything else	—	raises ValueError

The schema is a proposal — inspect it

The int-column guess (ordinal vs continuous vs a label-encoded nominal) is the one thing no extractor can know for sure; it also fires a UserWarning. Override any field before passing the schema on.

Never touches the model

model must already be a numpy->numpy callable. If your model consumes a DataFrame (e.g. an sklearn Pipeline), wrap it yourself — see effector.adapters.

Parameters:

Name	Type	Description	Default
df		a pandas DataFrame with numeric / bool / categorical / string columns (no missing values).	required
cat_limit	int	cardinality threshold for the int-column ordinal heuristic (default 10), recorded on the returned schema.	DEFAULT_CAT_LIMIT
return_encoding	bool	also return the DataFrameEncoding, whose transform(df_new) encodes later frames with the same codes.	False

Returns:

Type	Description
	(X, schema) where X is a (N, D) float64 numpy array and schema
	is an effector.Schema with feature_names, feature_types,
	cat_limit, and category_names populated; (X, schema, encoding)
	when return_encoding=True.

Raises:

Type	Description
TypeError	df is not a pandas DataFrame.
ValueError	a column contains missing values (the error names the column — impute or drop first), or has an unsupported dtype.

Source code in effector/ingestion.py

def from_dataframe(
    df, *, cat_limit: int = DEFAULT_CAT_LIMIT, return_encoding: bool = False
):
    """Extract a numpy matrix and a populated `Schema` from a pandas DataFrame — data only.

    ```python
    X, schema = effector.from_dataframe(df)
    pdp = effector.PDP(X, model, schema=schema)
    ```

    Constructors hard-reject a DataFrame as `data` (effector is numpy-only);
    this is the opt-in door. It reads column names, maps dtypes to feature
    types, encodes non-numeric columns to float codes, and records the level
    labels in `category_names` (a `{code: name}` dict per encoded column):

    | DataFrame dtype | feature type | encoded as |
    |---|---|---|
    | float | continuous | values as-is |
    | int | ordinal if `nunique < cat_limit`, else continuous | values as-is |
    | bool | ordinal | `False, True` → `0.0, 1.0` |
    | `Categorical`, ordered | ordinal | category codes |
    | `Categorical`, unordered | nominal | category codes |
    | object / string | nominal | codes of `astype("category")` |
    | anything else | — | raises `ValueError` |

    !!! warning "The schema is a proposal — inspect it"
        The int-column guess (ordinal vs continuous vs a label-encoded
        nominal) is the one thing no extractor can know for sure; it also
        fires a `UserWarning`. Override any field before passing the schema on.

    !!! note "Never touches the model"
        `model` must already be a numpy->numpy callable. If your model
        consumes a DataFrame (e.g. an sklearn Pipeline), wrap it yourself —
        see `effector.adapters`.

    Args:
        df: a pandas DataFrame with numeric / bool / categorical / string
            columns (no missing values).
        cat_limit: cardinality threshold for the int-column ordinal heuristic
            (default 10), recorded on the returned schema.
        return_encoding: also return the `DataFrameEncoding`, whose
            `transform(df_new)` encodes later frames with the same codes.

    Returns:
        `(X, schema)` where `X` is a `(N, D)` float64 numpy array and `schema`
        is an `effector.Schema` with `feature_names`, `feature_types`,
        `cat_limit`, and `category_names` populated; `(X, schema, encoding)`
        when `return_encoding=True`.

    Raises:
        TypeError: `df` is not a pandas DataFrame.
        ValueError: a column contains missing values (the error names the
            column — impute or drop first), or has an unsupported dtype.
    """
    if not is_dataframe(df):
        raise TypeError(
            f"from_dataframe expects a pandas DataFrame, got {type(df).__name__}"
        )
    cat_limit = _prep_cat_limit(cat_limit)
    matrix, names, inferred_types, categories, _dtypes, heuristic_idx = (
        _encode_dataframe(df, cat_limit)
    )

    # human-readable level names keyed by code, for every declared level — the
    # same record `DataFrameEncoding` replays, so names and codes cannot drift
    category_names = [None] * len(names)
    for j, enc in categories.items():
        category_names[j] = enc.level_names()
    if not any(n is not None for n in category_names):
        category_names = None

    # same guidance the numpy door gives: only int -> ordinal is a risky guess
    risky = [j for j in heuristic_idx if inferred_types[j] == ORDINAL]
    if risky:
        _heuristic_warning([(names[j], inferred_types[j]) for j in risky])

    schema = Schema(
        feature_names=names,
        feature_types=inferred_types,
        cat_limit=cat_limit,
        category_names=category_names,
    )
    if return_encoding:
        return matrix, schema, DataFrameEncoding(tuple(names), dict(categories))
    return matrix, schema

effector.ingestion.FeatureMetadata(feature_names, feature_types, cat_limit, target_name, scale_x_list=None, scale_y=None, category_names=None) dataclass

Resolved input metadata — what every effect object stores as feature_metadata.

pdp.feature_metadata.feature_names   # ['hr', 'temp', 'workingday']
pdp.feature_metadata.feature_types   # ['ordinal', 'continuous', 'nominal']

The read-only result of ingestion: Schema fields merged with inference, every gap filled. You never construct one — read it off a fitted object. Fields mirror Schema, except everything is resolved: types are canonical strings and category_names is a {feature_idx: {level_value: name}} map.