Hello Bioinformatics lovers,

Tommy here. Kids school starts this week. Ready to learn something new?

Twenty RNA-seq samples. Responders and non-responders to a drug.

Two ways to find what separates them, and one of them will fool you if you're not careful.

Option A: run the samples through an RNA foundation model, pull sample embeddings, feed those into logistic regression or lasso, validate with leave-one-out cross-validation.

You might see an AUC of 0.9. Everyone gets excited.

Here's the problem. With 20 samples and thousands of embedding dimensions, that number is fragile.

Leave-one-out on tiny cohorts overfits quietly, and this isn't just a hunch.

Studies on small-sample machine learning validation show the same pattern again and again: cross-validated performance estimates stay inflated in high-dimensional, low-sample-size settings, and the bias persists even as sample size grows into the hundreds.

The only test that counts is a dataset your model has never seen, and most 20-sample pilot studies don't have one.

Option B: conventional bioinformatics. Differential expression, pathway analysis, then a close look at specific genes. Slower, less flashy, but every step is interpretable.

Here's what surprised me running both on the same data: they often converge.

The genes driving the embedding separation show up in the DE list too. Different routes, same biology.

Sample size is the deciding factor. Machine learning needs enough samples to generalize, and at n=20 it doesn't have them.

Classical statistical modeling (e.g, DESeq2) was built for exactly this regime, and it hands you results you can explain to a biologist without waving at a black box.

Foundation models earn their keep at scale, on atlases with thousands of samples.

On a 20-sample pilot, a t-test and a volcano plot still do the job.

Before reaching for embeddings on a small cohort, ask:

• Do I have an external validation set?
• Can I explain the features driving the prediction?
• Would DE plus pathway analysis answer the same question?

No, no, yes: run the boring analysis first.

Have you run both approaches on the same small cohort? Reply and tell me what you found.

If you find the newsletter helpful, forward it to your friends so they can sign up here https://divingintogeneticsandgenomics.kit.com/profile​

Happy Learning!

Tommy aka crazyhottommy

PS:

If you want to learn Bioinformatics, there are four ways that I can help:

1. My free YouTube Chatomics channel, make sure you subscribe to it.
2. I have many resources collected on my github here.
3. I have been writing blog posts for over 10 years https://divingintogeneticsandgenomics.com/​
4. Lastly, I post daily on Linkedin https://www.linkedin.com/in/%F0%9F%8E%AF-ming-tommy-tang-40650014/recent-activity/all/​

Stay awesome!