Home » AI » ChemBERTa Statistics 2026: Chemical AI Model Performance And Research Data

ChemBERTa Statistics 2026: Chemical AI Model Performance And Research Data

ChemBERTa-77M-MLM recorded 49,475 monthly downloads on Hugging Face as of December 2025, making it one of the most downloaded transformer models built for computational chemistry. The model family, maintained by the DeepChem organization, now spans three major versions and processes SMILES molecular strings to predict chemical properties for drug discovery. This article covers the latest ChemBERTa statistics across model versions, benchmark performance, adoption data, and the broader AI-driven drug discovery market the model operates in.

ChemBERTa Statistics 2026 – TL;DR

ChemBERTa-77M-MLM had 49,475 monthly downloads on Hugging Face in December 2025.

ChemBERTa-3 pre-trained on 1.4 billion compounds from ZINC20, released in July 2025.

The model achieved 95.2% ROC-AUC on ClinTox toxicity classification, beating the Chemprop baseline at 90.5%.

Combined monthly downloads across all ChemBERTa variants reached 299,852 in 2024.

The AI drug discovery market was valued at roughly $2.9 billion in 2026, based on Fortune Business Insights and consolidated analyst estimates.

ChemBERTa is a family of RoBERTa-based transformer models trained on SMILES chemical strings. Built and maintained by Deep Forest Sciences through the DeepChem open-source library, ChemBERTa predicts molecular properties like toxicity, solubility, and blood-brain barrier penetration. The original model trained on 10 million PubChem compounds. ChemBERTa-2 scaled that to 77 million. ChemBERTa-3, published in Digital Discovery in 2026, pushed training to 1.4 billion compounds from the ZINC20 library. Researchers use ChemBERTa for molecular property prediction tasks, with the model and its weights freely available on Hugging Face for fine-tuning on specific pharmaceutical applications.

How Many Downloads Does ChemBERTa Have?

ChemBERTa models collected 299,852 combined monthly downloads across three primary Hugging Face repositories during 2024. The flagship ChemBERTa-77M-MLM variant alone recorded 49,475 monthly downloads as of December 2025. The MLM-pretrained variant gets roughly 10 times the traffic of the 10M-MTR model.

Model Variant	Pre-Training Data	Monthly Downloads (Dec 2025)
ChemBERTa-77M-MLM	77M PubChem compounds	49,475
ChemBERTa-10M-MLM	10M PubChem compounds	~5,000
ChemBERTa-10M-MTR	10M PubChem compounds	~4,500

Source: Hugging Face Model Hub

The DeepChem organization on Hugging Face has 91 followers. Seven derived fine-tuned models and three active Hugging Face Spaces use ChemBERTa as their base, based on data from late 2025.

ChemBERTa Model Architecture and Versions

ChemBERTa adapts the RoBERTa transformer for chemical language. The architecture uses 12 attention heads across 6 layers, producing 72 distinct attention mechanisms. It processes SMILES molecular strings rather than natural language text, with a vocabulary of 591 tokens and a maximum sequence length of 512 tokens.

Version	Release Year	Training Dataset	Compounds
ChemBERTa	2020	PubChem 10M	10 million
ChemBERTa-2	2022	PubChem 77M	77 million
ChemBERTa-3	2025	ZINC20	1.4 billion

Source: arXiv (ChemBERTa, ChemBERTa-2), ChemRxiv/Digital Discovery (ChemBERTa-3)

ChemBERTa Training Data Scale by Version

ChemBERTa-3, published in Digital Discovery (RSC) in early 2026 after a July 2025 preprint, uses both AWS-based Ray deployments and on-premise high-performance computing clusters. ZINC20 contains 1.4 billion compounds from 310 catalogs across 150 suppliers, designed specifically for virtual screening. All ChemBERTa-3 model weights are open-sourced.

ChemBERTa Benchmark Performance

ChemBERTa models are evaluated against the MoleculeNet benchmark suite, which includes tasks like toxicity prediction, blood-brain barrier penetration, and HIV replication inhibition. ChemBERTa-2 (MLM, 77M) outperformed the D-MPNN baseline on 6 of 8 MoleculeNet tasks, according to the ChemBERTa-2 paper.

Benchmark	Metric	ChemBERTa Score	Chemprop Baseline
ClinTox	ROC-AUC	0.952	0.905
Tox21 (SR-p53)	ROC-AUC	0.728	0.688
HIV	ROC-AUC	0.793 (MLM)	—
Lipo	RMSE	0.798	—

Source: arXiv ChemBERTa-2 paper (Ahmad et al., 2022)

ChemBERTa vs Chemprop Baseline (ROC-AUC)

The MLM pre-training approach outperformed multi-task regression by 6 percentage points on the HIV task, recording 0.793 AUROC vs 0.733 for MTR. ChemBERTa-MLM-100M also beat the larger MolFormer 1.1B model on BBBP and ClinTox benchmarks, despite having far fewer parameters.

Scaling pre-training data from 5 million to 77 million compounds produced a 25-35% reduction in loss and 5-10% improvement in RMSE or ROC-AUC on select tasks. A January 2026 study by Wang et al. showed that freezing ChemBERTa as a molecular encoder and projecting mass spectrometry embeddings into its latent space improved Recall@1 by 20-25 points over end-to-end neural methods.

ChemBERTa Pre-Training Methods Compared

ChemBERTa-2 tested two self-supervised pre-training strategies: masked language modeling (MLM) and multi-task regression (MTR). MLM masks 15% of tokens in each SMILES string and trains the model to reconstruct them. MTR predicts 200 physicochemical properties for each compound.

Pre-Training Method	HIV ROC-AUC	ClinTox ROC-AUC	Transfer Learning Quality
MLM (77M)	0.793	0.952	Higher
MTR (77M)	0.733	—	Lower

Source: arXiv ChemBERTa-2 paper (Ahmad et al., 2022)

MLM consistently produced better transfer learning results, which explains why the MLM variant on Hugging Face draws over 10 times the downloads of the MTR variant. A September 2025 paper from ChemRxiv on LoRA-based fine-tuning (EffiChem) reported that applying Low-Rank Adapters to ChemBERTa improved AUC by 3-5% across classification tasks while cutting trainable parameters substantially. This approach was tested on both ChemBERTa and MolFormer.

ChemBERTa Research Applications

ChemBERTa is used across several drug discovery tasks: molecular property prediction, toxicity screening, pharmacokinetics prediction, and drug-drug interaction classification. A January 2026 paper in Drug Discovery Today described a dual-stream ChemBERTa-GNN hybrid that outperformed MolBERT and classical QSAR methods on solubility, toxicity, and binding affinity prediction.

Attention visualization studies have shown that specific attention heads in ChemBERTa focus on functional groups like hydroxyl and carbonyl groups, while others specialize in aromatic ring structures. This interpretability has made ChemBERTa a useful tool for understanding what the model learns about molecular structure, beyond raw prediction accuracy.

The AlphaFold 2 protein structure prediction system and ChemBERTa address different parts of the drug discovery pipeline. AlphaFold predicts protein targets; ChemBERTa predicts small-molecule properties. A 2024 study from Leiden University (PCMol) combined AlphaFold protein embeddings with transformer-based molecular generation, showing these tools can be used together.

ChemBERTa vs Other Chemical Language Models

ChemBERTa competes with several other transformer-based molecular models. MolBERT, from BenevolentAI, trains on 1.6 million ChEMBL compounds and achieves comparable results with 48 times less data than ChemBERTa-2. MoLFormer, from IBM, trained on 1.1 billion molecules. NVIDIA’s MegaMolBART trained on approximately 1.45 billion molecules.

Model	Developer	Training Compounds	Architecture
ChemBERTa-2	DeepChem / Deep Forest Sciences	77 million	RoBERTa
ChemBERTa-3	Deep Forest Sciences / LLNL	1.4 billion	RoBERTa
MolBERT	BenevolentAI	1.6 million	BERT
MoLFormer	IBM Research	1.1 billion	Transformer
MegaMolBART	NVIDIA	~1.45 billion	BART

Source: Respective model papers via arXiv and ChemRxiv

Chemical Language Models by Training Data Size

ChemBERTa-3 is open-sourced entirely, unlike MoLFormer’s larger variants. MolBERT achieves strong results with far fewer training compounds due to chemistry-aware tokenization and auxiliary pre-training tasks. The choice between these models often depends on the specific task and available compute.

AI Drug Discovery Market Size

The AI drug discovery market, which ChemBERTa and similar models feed into, was valued at approximately $4.46 billion in 2025, according to Fortune Business Insights. That figure is expected to reach $5.0 billion in 2026 and $12.56 billion by 2034, growing at a 12.2% CAGR. A consolidated estimate from New Market Pitch, synthesizing multiple analyst reports, puts the 2026 figure closer to $2.9 billion under a narrower definition.

AI Drug Discovery Market Projection (USD Billions)

North America holds roughly 52-66% of the global AI drug discovery market, depending on the source. Pharmaceutical and biotech companies account for about 59% of the market by end-user share. The oncology segment leads therapeutic areas at 21% of revenue, while infectious diseases is the fastest-growing segment. Isomorphic Labs (Alphabet) held over 3.2% market share in 2025, per Global Market Insights, making it the single largest player. The broader AI ecosystem continues to accelerate adoption across the pharmaceutical industry.

ChemBERTa Technical Specifications

Parameter	Value
Base Architecture	RoBERTa (via Hugging Face)
Attention Heads	12
Transformer Layers	6
Attention Mechanisms	72
Vocabulary Size	591 tokens (SMILES characters)
Max Sequence Length	512 tokens
Pre-Training Objective (MLM)	15% token masking
Pre-Training Objective (MTR)	200 physicochemical properties
Fine-Tuning Tasks	MoleculeNet (BBBP, ClinTox, HIV, Tox21, Lipo)
License	Open-source (MIT via DeepChem)

Source: arXiv ChemBERTa and ChemBERTa-2 papers

FAQ

What is ChemBERTa used for?

ChemBERTa predicts molecular properties from SMILES chemical strings. Primary uses include toxicity screening, solubility prediction, blood-brain barrier penetration estimation, and drug-drug interaction classification in pharmaceutical research.

How many downloads does ChemBERTa get per month?

The flagship ChemBERTa-77M-MLM variant recorded 49,475 monthly downloads on Hugging Face as of December 2025. Combined downloads across all variants totaled 299,852 per month in 2024.

What is the difference between ChemBERTa-2 and ChemBERTa-3?

ChemBERTa-2 trained on 77 million PubChem compounds. ChemBERTa-3, released in 2025, expanded to 1.4 billion compounds from ZINC20 and added support for both AWS and HPC cluster training.

Is ChemBERTa open source?

Yes. All ChemBERTa versions are open-source through the DeepChem library and Hugging Face. Pre-trained weights are freely available for download and fine-tuning on custom molecular prediction tasks.

How does ChemBERTa compare to MolBERT?

ChemBERTa-2 trains on 77 million compounds while MolBERT uses 1.6 million. MolBERT achieves comparable results with 48 times less data thanks to chemistry-aware tokenization. Both use BERT-family architectures adapted for SMILES strings.