Hello Bioinformatics lovers

Tommy here. Today's newslettere is Sponsored by Bright Data. Bright Data turns any public website into a hosted data API from a single prompt.

=========

I typed one line into Claude Code: install the Bright Data CLI, my API key is in .zshrc, connect the MCP.

It handled the install, the config, all of it. I restarted Claude Code and the bright-data server showed up with its tools loaded.

Then I asked in plain English: find the top 5 most discussed AI papers today on Linkedin and save them to a markdown file. It did.

No proxy setup. No CAPTCHA handling. No browser fingerprinting. That layer is where my scraping projects usually die, and I never touched it.

If you want something durable instead of one-off, bdata scraper create builds a Scraper Studio scraper you own. When the site changes its HTML, one command heals it.

5,000 free credits a month, no card. [here]​

===========

Now, let's dive into today's newsletter.

A group fit a LASSO model to HapMap gene expression data and got a correlation of 0.92 between the observed response and the fitted values. In most papers that number ends the argument.

​Fan and Zhou et.al asked a different question. What correlation would you expect if the response were independent of every single covariate? They derived the distribution of the maximum spurious correlation, the best fit a selection algorithm can produce by chance alone when it searches across thousands of features.

In high dimensions that benchmark sits far higher than intuition suggests. A fit of 0.92 becomes a number you have to defend rather than a number that speaks for itself.

That example contains the whole problem with transcriptomics, proteomics and metabolomics.

Test 20,000 genes against a phenotype and some will track it.

Your data will hand you a clean, publishable story, and nothing in your pipeline will object until someone tries to replicate it.

Correct for every test you ran

Benjamini-Hochberg or Bonferroni is the floor. The part people skip is that m has to include every test you actually performed, including the ones you ran, looked at, and quietly set aside. A correction applied only to the analysis that survived controls nothing.

Penalize complexity

LASSO (L1), ridge (L2) and elastic net (both penalties) shrink coefficients toward zero and make the model spend its budget on features that earn it. Ridge keeps every feature at smaller weights.

LASSO drives some coefficients to exactly zero. Elastic net handles groups of correlated features better than LASSO alone, which matters when your genes are co-expressed.

Cross-validate the procedure, not the final model

This is where most published signatures break. Ambroise and McLachlan showed in 2002 that when you select genes on the full dataset and then cross-validate the classifier built from them, the error rate you report is biased downward.

The held-out samples helped pick the features, so they were never held out. Their fix is to run feature selection inside each training fold, from scratch, every time.

Feature selection gets recommended as a defense against overfitting. Run outside the CV loop, it produces the overfitting it was meant to prevent.

If a signature survives all of that, replicate it in an independent cohort. Nothing else settles the question.

I've written up the details on both halves of this:

👉 Understanding p value, multiple comparisons, FDR and q value​
👉 Regularization for scRNAseq marker selection​

What's the most convincing result you've watched fail to replicate? Hit reply and tell me. I read every one.

Happy Learning!

Tommy aka crazyhottommy

PS:

If you want to learn Bioinformatics, there are four ways that I can help:

1. My free YouTube Chatomics channel, make sure you subscribe to it.
2. I have many resources collected on my github here.
3. I have been writing blog posts for over 10 years https://divingintogeneticsandgenomics.com/​
4. Lastly, I post daily on Linkedin https://www.linkedin.com/in/%F0%9F%8E%AF-ming-tommy-tang-40650014/recent-activity/all/​

Stay awesome!