{"entity":{"id":"nucleotide-transformer","kind":"technology","name":"Nucleotide Transformer (InstaDeep)","aka":[],"tldr":"DNA language models trained on thousands of genomes for variant and regulatory prediction.","summary":"Nucleotide Transformer from InstaDeep is a family of DNA language models trained with masked language modelling over 6-mer tokens, learning sequence patterns without labels. The Nature Methods 2024 paper describes models up to 2.5B parameters trained on 3,200 human genomes and 850 species, and fine-tunes them for regulatory element and variant effect prediction. The weights are open, which has made the models a common baseline in genomic language modelling. Their context window is short compared with newer long-context models, so they cannot see distant regulatory interactions, and their usefulness for interpreting non-coding cancer variants depends on downstream fine-tuning. For a newcomer: it is an early, freely available model that learned patterns in DNA from thousands of genomes.","status":"emerging","asOf":"2026-09-08","links":[{"label":"Nature Methods 2024","url":"https://doi.org/10.1038/s41592-024-02523-z"}],"tags":["foundation-model","genome"],"related":[],"cancers":[],"sections":["ai-computation"],"technologies":[],"targets":[],"drugs":[],"companies":[],"institutions":[],"pathways":[],"terms":[],"trials":[],"people":[],"bottlenecks":[],"keyPapers":["paper-dalla-torre-nat-methods"],"journals":[],"dependsOn":[],"notes":[],"principle":"Nucleotide Transformer uses masked language modelling over 6-mer tokens.","strengths":["Open weights"],"limitations":["Short context"],"since":2024},"route":"/technologies/nucleotide-transformer/","neighbours":{"section":[{"id":"ai-computation","kind":"section","name":"AI & Computation","route":"/fronts/ai-computation/"}],"paper":[{"id":"paper-dalla-torre-nat-methods","kind":"paper","name":"Nucleotide Transformer: building and evaluating robust foundation models for human genomics","route":"/key-papers/paper-dalla-torre-nat-methods/"}],"term":[{"id":"autoregressive-modelling","kind":"term","name":"Autoregressive (next-token) modelling","route":"/terms/autoregressive-modelling/"},{"id":"genomic-and-protein-language-models","kind":"term","name":"Genomic and protein language models: Evo 2, Enformer, ESM","route":"/terms/genomic-and-protein-language-models/"},{"id":"variant-effect-prediction","kind":"term","name":"Variant effect prediction","route":"/terms/variant-effect-prediction/"}]}}