py-evoFE: Automated Evolutionary Feature Engineering for Tabular ML in Python (Genetic Algorithms + Scikit-Learn + Polars) [P]

[u/tanopereira]

Hey everyone!

I’m excited to announce the release of py-evoFE (v0.3.0) — an open-source Python library that uses genetic algorithms to automatically discover, combine, and optimize feature transformations for tabular datasets.

• GitHub: https://github.com/tanopereira/py-evoFE
• PyPI: pip install py-evoFE
• License: MIT

The Problem It Solves

Feature engineering is still where most tabular ML competitions and production models are won or lost. While GBDTs like LightGBM and XGBoost excel on raw tabular data, they struggle to discover complex ratios, nested group-by aggregations, nonlinear dimensional projections, and interaction graphs on their own.

Manual feature engineering is either tedious or constrained by human intuition, while brute-force feature generation explodes the feature space exponentially with colinear noise and high memory usage.

What py-evoFE Does

py-evoFE searches the space of possible feature recipes using genetic programming:

1. Hierarchical Chaining: Evolved features become building blocks for future generations (e.g., log(ratio(groupby_mean(x1, by=x2), x3))).
2. 40+ Built-in Transformers:
   • Non-linear arithmetic & log-ratios
   • Target encoding (multiclass, pooled, WoE, quantile target encodings)
   • String similarity (MinHash, Gap encodings)
   • Manifold & Dimensionality Reduction (PCA, UMAP, MCA, FAMD, Between-Group PCA)
   • Graph & Density Clustering (Genie, Lumbermark, MST anomaly scoring)
3. Performance & Speed:
   • Vectorized computation powered by Polars and PyArrow.
   • Matrix Hashing & Nearest-Neighbor Caching: Stateful projections (like UMAP and $K$-NN lookups) are cached via byte-hashing to eliminate redundant computation across CV folds.
   • Multi-Fidelity Screening: Fast low-fidelity CV screens initial populations; only promising candidates proceed to full-fidelity evaluation.
4. Island Model & Caruana Ensembling:
   • Multi-population parallel search across Ring, Torus, Grid, Hypercube, and Tiered topologies with Gibbs migration.
   • Post-search greedy Caruana ensembling over island winners' out-of-fold predictions.
5. Interactive Replay Viewer:
   • Run view(evo.get_recipe()) to generate a self-contained, zero-dependency HTML dashboard replaying the evolutionary search over time.
6. 100% Scikit-Learn Compatible:
   • Implements fit, transform, predict, and predict_proba. Plugs directly into standard sklearn.pipeline.Pipeline and GridSearchCV.

---

Quick Example

import polars as pl
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from evofe import EvoFE

# Load data
bc = load_breast_cancer(as_frame=True)
df = pl.from_pandas(bc.frame)
X, y = df.drop("target"), df["target"].to_numpy()

X_train, X_test, y_train, y_test = train_test_split(
    X.to_numpy(), y, test_size=0.2, random_state=42, stratify=y
)
X_train_df = pl.DataFrame(X_train, schema=X.columns)
X_test_df = pl.DataFrame(X_test, schema=X.columns)

# 1. Initialize EvoFE
evo = EvoFE(
    task="classification",
    evaluator="lightgbm",       # "lightgbm" | "xgboost"
    pop_size=15,
    n_generations=10,
    cv_folds=3,
    verbose=True,
    random_state=42
)

# 2. Fit: Runs evolutionary search
evo.fit(X_train_df, y_train)

# 3. Inspect evolved recipe
recipe = evo.get_recipe()
print(f"Discovered {len(recipe.genes)} high-impact features:")
for gene in recipe.genes:
    print(f" • {gene.to_formula()} -> {gene.output_col}")

# 4. Transform & Predict
preds = evo.predict(X_test_df)
proba = evo.predict_proba(X_test_df)

---

Why not just brute-force feature generation?

Brute-force libraries generate thousands of features upfront, leading to severe overfitting, massive memory usage, and colinear noise that degrades tree-based models. py-evoFE uses evolutionary selection pressures with complexity penalties to discover compact, parsimonious recipes that actually improve generalization.

I’d love for the community to try it out on your datasets or Kaggle benchmarks! Feedback, issues, and feature requests are very welcome on GitHub.

[u/Gere1]

Tbh, it's a bit questionable if you don't provide Kaggle benchmarks yourself. Ideally you'd provide a link to a Kaggle notebook where you demo your solution where it reaches a score above other simple solutions (without putting in extra data science modifications). If it the project is so automatic, why did you not test it on Kaggle? Most grand claims about automatic tabular data science turn out to be void.

[u/tanopereira → u/Gere1]

I purposefully avoided it as then I'd have been "accused" of cherry picking, the typical problem with benchmarks.

Still, there are several things I believe the package does that are helpful. Setting pop_size=1 and generations=1 and using a tuned model allows for a one liner optuna optimized model.

It does feature selection/masking, has dynamic penalties etc.

This is without using the feature engineering part.

[u/Gere1 → u/tanopereira]

If you pick some random dataset from the internet, it looks like cherry picking. You could pick the most obvious benchmark that others know of so that it's not your pick. Go to Kaggle, filter for tabular leaderboards, pick the largest 3 (which do not saturate at 99%). Use standard parameters in your model and don't put work into it. That would be convincing. It's much more dodgy if you don't show any benchmark. Shouldn't that be easy with an automated model?