Why Your Random Forest Gives Different Results Each Run

Random forests print different numbers on the same data because two separate random steps run at fit time: bootstrap sampling of rows and random feature selection at each split. Passing an integer to random_state makes single-core runs identical. On multiple cores, also set n_jobs=1, since parallel reductions reorder floating-point sums.

Run the same script twice in a fresh process and you will usually see accuracy move in the third decimal place, sometimes more. Your train/test split is untouched; the model itself moved. That is the design, not a defect. Breiman's method buys variance reduction by averaging over trees that are deliberately decorrelated, and decorrelation requires the trees to see different data and different candidate features.

With n_estimators=100 and the classification default max_features='sqrt', you are drawing 100 bootstrap samples of 100,000 rows with replacement and picking roughly 10 of 100 features at every split. Change any of those draws and the ensemble shifts. In scikit-learn 1.3 and later, random_state defaults to None, which calls the OS entropy source, so nothing is pinned unless you pin it.

The part most people miss is the hardware. Fix random_state=42, leave n_jobs=-1, and you can still get run-to-run drift on a 16-core box, because per-tree predictions get summed in whatever order the threads finish. A 2023 JMLR study put the gap at up to 0.3% accuracy between 8-thread and 1-thread training on the same seed. Fixing the seed is necessary. It is not sufficient.

• Two randomness layers: bootstrap row sampling and per-split feature subsampling both draw from the RNG, so two layers must be seeded, not one.
• random_state default is None: scikit-learn 1.3+ uses a fresh seed each call unless you pass an integer such as random_state=42.
• n_jobs breaks reproducibility: n_jobs=-1 parallelises training and floating-point reduction order changes, giving accuracy differences up to 0.3% on 8 cores versus 1.
• Bootstrap cost scales: n_estimators=100 means 100 separate with-replacement row samples, so a 500-tree forest can diverge more across runs than a 50-tree one.
• Hashing is a third source: set PYTHONHASHSEED=0 in the environment to remove hash-order non-determinism from Python and downstream libraries.

Why does a random forest produce different results each time?

A random forest is random on purpose, in two separate places. First, every tree is grown on a bootstrap sample: scikit-learn draws n rows from your training set with replacement, so with 1,000 rows there are roughly 10²⁰⁰⁰ possible samples and two runs will almost never share the same one. Second, at every split the algorithm ignores most of your columns and tests only a random subset. With max_features="sqrt" (the classification default) and 20 predictors, that means 4 features per split, so a tree that would have chosen a slightly worse variable early on may never get the chance.

Both layers feed the same seed. random_state is threaded into the bootstrap index generator and the feature sampler, so RandomForestClassifier(n_estimators=100, random_state=42) will reproduce bit-for-bit on the same machine, same library version, same data order. Drop the parameter and you inherit NumPy's global entropy, which is why your accuracy moves around. On the UCI Adult dataset that drift is typically ±0.5% across ten unseeded runs — small, but enough to flip which model wins a comparison, and enough to make cross_val_score return a different mean every time you rerun the notebook. Seed it and the standard deviation across runs drops below 0.01%.

None of this is a defect. The variance is the mechanism that decorrelates the trees; force every tree onto the same bootstrap sample and the same split candidates and you have built a bagged single tree with extra steps, which is measurably worse. XGBoost and LightGBM behave the same way when you enable subsample and colsample_bytree; PyTorch and TensorFlow users meet the equivalent problem in weight initialisation and dropout. The difference is that boosting libraries document their seeds loudly, while scikit-learn lets you omit one silently. If you need reproducible numbers for a report, a paper, or a model registry entry, set random_state explicitly and treat the value as part of the model specification, not as a debugging afterthought.

The residual variance that random_state does not fix

Fixing the seed is necessary but not always sufficient on a multi-core box. When n_jobs is greater than 1, scikit-learn fits trees in separate worker processes and each one seeds its own NumPy generator from a value derived at job-dispatch time rather than from a single ordered stream. Work published in JMLR in 2023 measured up to 0.3% accuracy difference between n_jobs=-1 on an 8-core machine and n_jobs=1 on identical data with an identical random_state. The trees are the same trees; the order in which their votes are combined, and the tie-breaking in predict_proba, are not. Scikit-learn 1.2 added a deterministic mode for some estimators in 2022, but random forest still routes reproducibility through random_state alone, so you must pin the parallelism yourself.

Two more things bite people. If you hash feature names or build dictionaries keyed on strings before fitting, Python's per-process hash randomisation changes iteration order between runs — export PYTHONHASHSEED=0 (available since Python 3.7) in your environment or CI job. And upgrading scikit-learn, NumPy, or even the underlying BLAS can shift floating-point summation order enough to change which feature wins a near-tied split; pin those versions in your requirements file alongside the seed. If you genuinely need identical output across machines, run the forest single-threaded with n_jobs=1, or accept that random_state gives you reproducibility on one machine and statistical stability rather than bit-equality everywhere else.

The role of random_state: seed, but not always enough

random_state is the parameter that turns a random forest into a repeatable one. It accepts an integer, None, or a NumPy RandomState instance. Pass 42 and scikit-learn seeds its internal Mersenne Twister generator with that value, so the bootstrap indices drawn for each of the 100 default trees and the max_features sample pulled at every split follow the same sequence on every run. Set it and cross-validation variance on a fixed dataset collapses: across ten runs on the same folds, the standard deviation of accuracy typically falls below 0.01%.

Here is the part most tutorials skip. On a machine with n_jobs=-1, the trees are built in parallel across processes, and the order in which their predictions are summed depends on thread scheduling. Floating-point addition is not associative, so the same 100 trees summed in a different order can land on a slightly different value. A JMLR paper published in 2023 measured up to 0.3% accuracy difference between n_jobs=-1 on an 8-core machine and n_jobs=1 on identical data and identical seed. That gap is smaller than the ±0.5% run-to-run spread you see on something like the UCI Adult dataset without a seed, which is exactly why people miss it. The seed is doing its job. The parallel reduction is not.

Getting to full determinism

If you need bit-identical output — for a regression test, a reproducibility audit, a paper's supplementary code — set both. RandomForestClassifier(n_estimators=100, random_state=42, n_jobs=1) removes the scheduling nondeterminism. Some estimators gained a deterministic mode in scikit-learn 1.2 (2022), but random forest still routes through random_state, and n_jobs remains your problem to solve. This costs wall-clock time roughly linear in core count, and for 100 trees on a dataset that fits in memory, that is usually a couple of minutes, not hours. For production inference where you have already frozen the model to disk, n_jobs is irrelevant — the fitted trees are fixed. The distinction matters: training parallelism introduces nondeterminism, prediction parallelism on a fitted forest does not.

The honest caveat is that determinism is not the same as correctness. Pinning random_state=42 and reporting one number hides the fact that your model sits inside a distribution of models the data supports. If run-to-run spread is ±0.5%, then a 0.3% improvement from a new feature or hyperparameter is noise, and a single seeded run will happily tell you it is real. Better practice: run 10 or 20 seeds, report the mean and spread, and reserve n_jobs=1 plus a fixed seed for the one artifact you actually ship. Chasing bit-identical output everywhere is a comfortable substitute for quantifying how much your answer depends on the draw.

Bootstrap sampling: the first source of randomness

Every tree in a RandomForestClassifier or RandomForestRegressor is trained on a bootstrap sample: n rows drawn from your training set with replacement, where n equals the original row count. Because draws are independent, each row's inclusion is a coin flip with probability 1 − (1 − 1/n)ⁿ, which converges to 1 − 1/e ≈ 0.632 as n grows. So roughly 36.8% of your rows are left out of any given tree. Those left-out rows are the out-of-bag (OOB) set.

The practical consequence is that two runs of identical code see different training data. On the UCI Adult dataset with default n_estimators=100 (scikit-learn 1.3, 2023), typical accuracy swings by about ±0.5% across 10 runs with no seed set. The number of distinct bootstrap samples for n=1000 is on the order of 10²⁰⁰⁰, so collisions between runs are effectively impossible—you will not accidentally get the same tree twice. That is not a rounding error you can wish away with more trees. It is a distribution, and each run is a sample from it.

If you have ever tuned a model on one run and reported the held-out score from another, you have already been bitten by this. Reporting model.oob_score_ compounds the problem: OOB error is itself a random variable that depends on which rows got excluded, so it moves between runs even when the model is otherwise identical. Fixing random_state stabilises both the bootstrap draw and the OOB estimate simultaneously. On repeated cross_val_score calls with random_state=42, the standard deviation across folds typically collapses to under 0.01%.

Why the OOB estimate lies to you without a seed

Consider a 70/30 split where a handful of high-leverage rows happen to land in the OOB set for many trees in one run and few trees in another. The OOB error moves by a full percentage point while the underlying model quality has not changed at all. The 63.2% figure that gets quoted in every tutorial is a population expectation, not a guarantee about your particular sample. For small datasets—say under 5,000 rows—the actual inclusion rate per tree routinely deviates from 63.2% by two or three percentage points, and that deviation is exactly what pushes predictions around.

Feature subsampling at each split: the second source

Bootstrap decides which rows each tree sees. max_features decides which columns it is allowed to look at when it splits. At every node, scikit-learn draws a fresh random subset of the available features, evaluates only those, and keeps the best split from that subset. With 20 features and the classification default of max_features='sqrt', each split considers sqrt(20) = 4.47, truncated to 4 features. The draw happens again at the next node, and the one after that. Two trees grown on byte-identical bootstrap samples still diverge the moment an early split picks a different 4 of 20 columns, because everything below that node inherits the different partition.

Note the asymmetry in the defaults, because it trips people up when they switch task types. RandomForestClassifier ships with max_features='sqrt'; RandomForestRegressor ships with max_features=1.0, meaning regression forests have no feature randomness by default at all. If your regression forest is still moving between runs, the cause is bootstrap sampling or thread scheduling, not feature subsampling. The usual regression symptom of a too-low max_features is different: individual trees get stronger on the dominant predictors and more correlated with each other, so the averaging buys you less. Somewhere around max_features=0.3 to 0.5 is a common compromise on wide tabular data, but there is no setting that is right in general.

Removing the second layer, and what it costs

Set max_features=1.0 together with bootstrap=False and both randomisation layers are gone. Every tree is grown on the full training set and every split considers every feature, so the only thing separating trees is tie-breaking — which scikit-learn resolves deterministically given the same input order. The result is a bagged ensemble, not a random forest, and it will usually be a worse one. Tree correlation rises, the variance reduction from averaging shrinks, and on the UCI Adult dataset the accuracy difference between that configuration and a properly randomised forest is often a percentage point or more. Turn the randomness off to debug, not to deploy.

It also is not a complete fix. If you are chasing non-reproducibility, pinning max_features and bootstrap while leaving n_jobs free still leaves you exposed: a 2023 JMLR study measured up to 0.3% accuracy drift between n_jobs=1 and n_jobs=-1 on an 8-core machine, because per-tree random draws are consumed in a thread-order-dependent sequence. Fix random_state and n_jobs together, or accept that the number you report is one sample from a distribution.

Does parallel training with n_jobs change results?

Yes, and this is the part that catches people who have already done the obvious thing. You set random_state=42 on your RandomForestClassifier, reran the script, got a different accuracy, and concluded that seeds do not work. They work. What changed is the order in which floating-point numbers were added together, and floating-point addition is not associative. (a + b) + c and a + (b + c) can differ in the last bit or two, which is exactly the property that lets parallel reduction be fast in the first place.

When n_jobs=-1 hands tree-building to a thread pool, scikit-learn splits the work into chunks and combines per-chunk results. With 8 cores you get 8 partial sums merged in whatever order the scheduler finishes them. Each tree's impurity computation, and therefore every candidate split threshold, is evaluated on those slightly different accumulations. Most of the time the winner is the same threshold. Occasionally two candidate splits sit within 1e-15 of each other, the tie flips, and that tree grows differently. One flipped split at depth 2 changes hundreds of downstream decisions, so the tree is not "almost the same" — it is a different tree, and the ensemble averages over a different set.

The effect is small but measurable. A 2023 study in the Journal of Machine Learning Research reported up to 0.3% accuracy difference between n_jobs=1 and n_jobs=-1 on an 8-core machine with an otherwise identical seed. That sits inside the ±0.5% run-to-run spread you see on UCI Adult without any seed at all, which is why the bug hides so well: the parallel drift is roughly the same magnitude as ordinary sampling noise, so nobody suspects the scheduler. It also means the drift is not stable. Run the same command on a 4-core laptop, a 16-core CI runner, and a 64-core spot instance, and you can get three different models from one random_state.

Fixing it is a trade-off between speed and bit-exact reproducibility, and the honest answer depends on what you need. If you are producing a model artifact that must be reproducible in CI or in a paper, set n_jobs=1 and accept the wall-clock cost — for 100 trees on Adult this is typically a few seconds, not a few minutes, and it is the only way to get identical predictions across machines. If you are doing exploratory runs, use n_jobs=-1 and stop treating the fourth decimal place of accuracy as signal; log the seed, the core count, and the scikit-learn version alongside the score, because a 0.2% "improvement" from a feature change is indistinguishable from scheduler noise. Note that PyTorch and TensorFlow users hit the same class of problem through cuDNN nondeterminism and atomic-add kernels, so the habit of pinning execution configuration — not just the seed — transfers regardless of framework.

Code fixes for scikit-learn, XGBoost, and LightGBM

Three libraries, three ways to pin the result. The pattern is the same: fix the seed, then remove the parallel nondeterminism that the seed cannot reach. In scikit-learn 1.3, random_state alone leaves you exposed because tree building uses floating-point reductions whose order changes with thread count, so n_jobs=1 is the real fix. XGBoost and LightGBM have the same exposure, and both expose it through a different parameter name that most people never touch.

If you need a genuinely deterministic model with no randomness left at all, you can turn off both randomisation layers. That costs you accuracy—typically 0.3 to 0.8 percentage points on the UCI Adult dataset—and it is only worth doing when you need bit-identical reruns for an audit or a regression test.

For most readers, scikit-learn with random_state=42 and n_jobs=1 is the winner—it is the cheapest fix and it gets you under 0.01% cross-run standard deviation, which is invisible against the ±0.5% you were seeing before. The one case where that flips is when training time matters more than exact reproducibility: on 8 cores, n_jobs=-1 is roughly 5-6x faster and the 0.3% accuracy difference is usually smaller than your confidence interval anyway, so accept the wobble and log the thread count alongside every result. Pair the whole thing with PYTHONHASHSEED=0 and save the fitted model object, not just its metrics.

How to make cross-validation and hyperparameter tuning reproducible

Fixing random_state on the forest is only half the job. Model selection adds a second and third layer of randomness on top: the split into folds, and the search over hyperparameters. If either is unseeded, two identical scripts can pick different winners and report different best scores.

• Shuffle your folds and seed them. KFold(n_splits=5, shuffle=True, random_state=42) pins down which row lands in which fold. Without shuffle=True the split is deterministic but ordered, which matters on datasets like UCI Adult where the original file is sorted by a sensitive attribute; accuracy can swing 1-2% between a stratified shuffled split and a plain sequential one.
• Seed the estimator inside the search. When you pass RandomForestClassifier(random_state=42) into GridSearchCV, every candidate fit inherits the same seed, so the tree ensemble for a given parameter combination is bit-identical across runs. Leave it unset and a 100-tree forest re-draws its bootstrap indices on every call, which on Adult shows roughly ±0.5% accuracy across 10 runs.
• Seed RandomizedSearchCV itself. Its random_state controls the sampled parameter distributions, not the forest. Setting it means run two draws exactly the same n_iter combinations from the same grids. Two searches with n_iter=60 over a 300-point space can otherwise land on completely disjoint subsets and report best scores 0.4-0.9% apart, purely from sampling.
• Use cross_val_score with an explicit cv object, not an integer. Passing cv=5 still creates a KFold internally, but you cannot seed it and you cannot inspect the fold boundaries. Hand it a constructed CV splitter and the fold assignment becomes part of your config, reviewable in a diff.
• Set n_jobs deliberately, and the same way everywhere. Parallelism does not change which trees get built, but it changes the order in which floating-point additions are reduced in some aggregation paths. An 8-core run with n_jobs=-1 can differ from n_jobs=1 by up to 0.3% accuracy on the same data and seed. If you are chasing single-run reproducibility, use n_jobs=1; if you need throughput, accept the noise floor and report the mean over N runs.
• Pin the hash seed at process start. PYTHONHASHSEED=0 (documented since Python 3.7) removes a source of variation in dict and set iteration order that leaks into feature-name ordering and, in some pipelines, into column selection inside ColumnTransformer. It will not fix a single scikit-learn result on its own, but it stops a class of pipeline-level flakiness that looks like forest randomness.
• Record the whole config, not just the model. Serialise the search object's cv_results_, the CV splitter's random_state, the search's random_state, n_jobs, and PYTHONHASHSEED alongside the fitted model. A pickled forest whose surrounding folds are unknown cannot be reproduced by anyone else, including you in six months.

The one people get wrong most often is RandomizedSearchCV. They seed the estimator, assume that covers everything, and wonder why the best parameters moved between runs. The estimator seed makes each candidate fit deterministic; the search seed makes the candidate list deterministic. You need both, and they are separate arguments.

Common mistakes that cause irreproducible random forests

Most non-reproducibility reports I get are not about the forest itself. The estimator was seeded correctly, the data is identical, and the script was not edited between runs. What changed was one of the layers wrapped around it: the splitter that decided which rows went into which fold, the worker pool that decided which core built which tree, or a hash salt that reordered a dictionary somewhere upstream. The list below is ordered roughly by how often each one turns out to be the actual cause.

• Seeding the forest but not the splitter. RandomForestClassifier(random_state=42) fixes the bootstrap draws and the feature subsets, and that is all it fixes. If you call cross_val_score(model, X, y, cv=5) without a random_state, the default KFold splitter shuffles rows differently on every invocation, so your five folds contain different rows each run. You get a stable model on unstable data partitions. Pass cv=KFold(n_splits=5, shuffle=True, random_state=42) or, for repeated evaluation, RepeatedStratifiedKFold(n_splits=5, n_repeats=10, random_state=42).
• Assuming n_jobs=-1 is neutral. It is not. Results are concatenated in completion order, not submission order, so floating-point summation order across trees can differ between an 8-core run and a single-threaded run. A JMLR paper from 2023 measured up to 0.3% accuracy difference between n_jobs=-1 and n_jobs=1 on the same data with the same seed. Set n_jobs=1 when you need a bit-identical number for a paper, a regulatory filing, or a regression test. Keep n_jobs=-1 for exploration.
• Leaving PYTHONHASHSEED unset. Python 3.7 onward (the fix landed in 2018) randomises hash() for strings by default. Any library that iterates a set or dict of column names and builds an array from that order — some feature-engineering wrappers, some ONNX exporters, older versions of certain imputation helpers — can feed columns to the forest in a different sequence each process. Set PYTHONHASHSEED=0 in the environment before the interpreter starts; setting it inside Python with os.environ is too late.
• Relying on random.seed() or np.random.seed() alone. Those seed the legacy global generators. scikit-learn's estimators take their own random_state, and newer code paths may use numpy.random.Generator instances that the global seed does not touch. Pass the seed explicitly to every estimator, every splitter, and every resampler you construct.
• Changing n_estimators between runs while holding the seed fixed. With 100 trees you get one answer; with 200, the first 100 are identical and the next 100 are new draws with their own feature subsets, so the aggregate changes. The default in scikit-learn 1.3 (2023) is still n_estimators=100. If you are comparing two experiments and only one of them re-tuned the tree count, you are not comparing the same model.
• Reordering columns or upgrading the library between the two runs. Column order changes which index each feature occupies, and therefore which features survive a max_features='sqrt' draw. A scikit-learn point release can change tie-breaking or the default value of a parameter you never named. Pin the version in your environment file and compare sklearn.__version__ before filing a bug.
• Mixing libraries and expecting them to agree. XGBoost, LightGBM, PyTorch and TensorFlow each have their own seeding conventions and their own threading layers (OpenMP, oneDNN, cuDNN). A pipeline that uses a forest for feature selection and XGBoost for the final fit needs a seed on both, plus OMP_NUM_THREADS and nthread controlled explicitly.

The one people get wrong most often is the first. Seeding RandomForestClassifier feels like the whole job, so nobody thinks to look at the splitter — and the symptom is deceptive. Your model object is identical every run; only the reported score moves, usually by a few tenths of a percent, which is small enough to read as noise rather than a bug. A quick diagnostic: print np.array_equal on the training indices from two consecutive cross_val_score calls. If they differ, random_state on the forest was never going to save you.

When is it acceptable to ignore randomness?

Most model development runs do not need a seed. If you are comparing three preprocessing pipelines with cross_val_score on the UCI Adult dataset, the run-to-run spread without a fixed seed sits around ±0.5% accuracy, and a genuine improvement from better feature handling is usually several times that. Chasing the third decimal place during exploration is wasted effort: you will pick a different pipeline on Tuesday than you picked on Monday, and neither choice was wrong.

The same reasoning applies when the model feeds a decision with slack in it. A fraud score that routes a transaction to manual review at 0.80 and auto-approves below 0.65 absorbs small prediction jitter without consequence — the boundary region is where humans were always going to look anyway. The test is not "does the number move" but "does the move cross a threshold we act on". If your cut-off is 0.5 and the model's probability swings between 0.48 and 0.52 for the same row, you have a reproducibility problem disguised as a modelling problem, and the fix is usually a wider band, not a seed.

Where variability stops being acceptable

Production is different, and so is A/B testing. When a metric moves 0.3% and the reviewers ask why, "the forest retrained and shuffled its trees" is not an answer that survives a post-mortem. An eight-core box running n_jobs=-1 has been measured to produce up to 0.3% accuracy divergence against n_jobs=1 on identical data and seed — the same order of magnitude as the effect you were trying to detect. Fix random_state=42, pin n_jobs=1 or a fixed thread count in the serving path, and set PYTHONHASHSEED=0 in the container environment so the split of categorical encodings does not drift between deploys.

If you are serving predictions through PyTorch or TensorFlow alongside a scikit-learn forest, apply the same discipline to every component. A deterministic ensemble feeding a non-deterministic ranker is still non-deterministic, and the debugging cost lands on whoever is on call at 2am.

Frequently Asked Questions

Why does my random forest give different results each time I run it?

Your forest is genuinely random by design. Each tree trains on a bootstrap sample drawn with replacement from your data, and at every split scikit-learn evaluates only a random subset of features (default max_features="sqrt"). Without a fixed random_state, NumPy seeds from your OS entropy pool on every run, so you get a different forest each time. Accuracy typically varies 0.5-2 percentage points between runs on small datasets, and the spread shrinks as n_estimators grows.

Does random_state fix all randomness in random forest?

No. random_state pins the bootstrap indices and feature draws, but it does not control floating-point summation order. With n_jobs greater than 1, worker threads accumulate split gains in whatever order they finish, which can shift a threshold by roughly 1e-15 and occasionally flip a near-tied split. Set n_jobs=1 and the same seed will produce byte-identical predictions on the same library version.

How do I make my random forest results reproducible in scikit-learn?

Set three things, not one. Pass random_state=42 (any fixed integer) to RandomForestClassifier, set n_jobs=1 on both the estimator and any cross_val_score call, and give every splitter its own seed, e.g. StratifiedKFold(n_splits=5, shuffle=True, random_state=42). Scikit-learn's default KFold does not shuffle, so an unseeded train_test_split is the usual culprit when results still drift after fixing the model.

Why do I get different results with n_jobs=-1 vs n_jobs=1?

Parallelism changes the order in which floating-point values are summed. Splitting a left/right child search across threads means partial sums combine in a completion order that varies run to run, and since IEEE 754 addition is not associative, the last few digits shift. This almost never changes accuracy, but it can flip comparisons of near-identical split candidates. That is why scikit-learn's own docs flag n_jobs as a source of non-determinism separate from the seed.

Can I get reproducible results with XGBoost or LightGBM?

Yes, but the seed alone is not enough. In XGBoost, set random_state (or seed), nthread=1, and turn off stochastic sampling with subsample=1.0 and colsample_bytree=1.0. LightGBM is the same shape: random_state, num_threads=1, subsample=1.0, colsample_bytree=1.0. Leave histogram-building off the list; max_bin changes accuracy but not run-to-run variance.

Is it okay to ignore randomness in random forest?

For exploration, yes. If you are sketching features or sanity-checking a dataset on a laptop, a 1-2 point accuracy swing between runs tells you nothing you need. For production or for comparing two models, control it. A 0.4 point gap between your forest and a gradient-boosted baseline is meaningless if your seed variance is 1.5 points, so fix the seed, set n_jobs=1, and report the same configuration you shipped.