PaLM

Google DeepMind ·

Verified

3 of 3 published values traced to a source3 primary sources, 3 archived1 not researched yet

Why this status

Cited to the lab's own announcement and the technical paper. Not verified because the context window could not be found in any cited primary source — the value may be correct, but this dataset cannot yet show where it came from. Capabilities evidenced from the primary sources: multilingual: “PaLM was trained using a combination of English and multilingual datasets that include high-quality web documents, books, Wikipedia, conversations, and GitHub code.” | coding: “PaLM 540B shows strong performance across coding tasks and natural language tasks in a single model, even though it has only 5% code in the pre-training dataset.”.

What these states mean · How records are judged · Has this record changed?

Google's 540-billion-parameter model that demonstrated breakthrough few-shot scaling.

codingmultilingualflagship

Details

ReleasedApril 4, 2022
stated in research.google

PaLM was released on April 4, 2022.

CompanyGoogle DeepMind
FamilyPaLM
Lineagebefore PaLM 2
EraPre-ChatGPT scaling
TypeLanguage model · LLM
ModalitiesText → Text
CapabilitiesCoding, Multilingual
WeightsProprietary
LicenceNot applicable — proprietary
Context window2K tokens
stated in arxiv.org

PaLM supports a context window of 2,048 tokens.

Parameters540B
stated in research.google, arxiv.org, arxiv.org

PaLM has 540,000,000,000 parameters.

Current information, elsewhere

This record is a historical one: every figure above is traced to a dated primary source. For what is true today — live pricing, current rankings, availability — these are better maintained than anything here could be.

Sources

Record status: verified · confidence 90/100 · 3 of 3 primary

Cited to the lab's own announcement and the technical paper. Not verified because the context window could not be found in any cited primary source — the value may be correct, but this dataset cannot yet show where it came from. Capabilities evidenced from the primary sources: multilingual: “PaLM was trained using a combination of English and multilingual datasets that include high-quality web documents, books, Wikipedia, conversations, and GitHub code.” | coding: “PaLM 540B shows strong performance across coding tasks and natural language tasks in a single model, even though it has only 5% code in the pre-training dataset.”.

Primary means published by the organisation that made the model. Verified records require at least one.

PaLM family

  1. PaLM April 4, 2022
  2. PaLM 2 May 10, 2023

Browse models

Compare PaLM with another model →
See PaLM on the interactive timeline →