Hugging Face: the hub and libraries for open AI models

Overview

Hugging Face is a company and an online platform at the centre of open machine learning. Its best-known product is the Hugging Face Hub, a website and service where people publish models, datasets and small demo applications called Spaces. Alongside the Hub, the company maintains a family of open-source libraries, led by Transformers, that make it easy to download a model, run it, fine-tune it and share the result.

For many practitioners, Hugging Face is the default first stop when they need a pretrained model. A developer can search for a sentiment classifier, a speech recogniser such as Whisper or an open language model, read its model card, try it in the browser and then load it in a few lines of Python. The same tools let organisations host private models and datasets for their own teams.

A good everyday picture is a combination of an app store and a public library. The app store part lets you find, try and install things quickly. The library part keeps careful records: who made each item, how it was built, what licence applies and which version you borrowed.

What it is

Hugging Face is easiest to understand as a platform plus a set of libraries.

Part	What it provides
Hub	Git-based repositories for models, datasets and Spaces, with versioning, model cards and access control
Model cards and dataset cards	Documentation files describing purpose, training data, limitations and licence
Spaces	Hosted demo apps, often built with Gradio
Transformers	A unified Python API for thousands of pretrained model architectures
Datasets	Loading, streaming and processing datasets efficiently
Tokenizers	Fast tokenisation, including BPE and related schemes
PEFT	Parameter-efficient fine-tuning methods such as LoRA
TRL	Post-training methods such as SFT, DPO and reinforcement learning
Accelerate	Running the same training code on one GPU, many GPUs or several machines
Diffusers	Pretrained diffusion models for images, audio and video
safetensors	A weight file format that stores tensors without executing code when loaded
`huggingface_hub`	The client library for uploading, downloading and managing repositories

Other projects include smolagents for building agents, and hosted inference services that run models without the user managing servers.

Where it came from

Hugging Face was founded in 2016 in New York by three French entrepreneurs: Clément Delangue, Julien Chaumond and Thomas Wolf. Its name and logo come from the hugging-face emoji. The company began with a chatbot app aimed at teenagers. After it open-sourced the model behind the chatbot, it turned towards building tools for the machine learning community.

The turning point was a library released in 2018 to make Google's BERT easy to use in PyTorch, first called pytorch-pretrained-bert. As more architectures were added, it was renamed pytorch-transformers and then simply transformers in 2019. The team described it in a demonstration paper at EMNLP 2020.

Further milestones followed:

December 2021: Hugging Face acquired Gradio, the library behind many Spaces demos.
2021 to 2022: it launched the BigScience research workshop with other research groups, which in 2022 released BLOOM, a multilingual open language model with 176 billion parameters.
April 2025: it acquired Pollen Robotics, a French robotics start-up, extending its open-source work into robotics.
December 2025: Transformers version 5 was announced. It made PyTorch the library's single backend, phasing out TensorFlow and Flax support, and the announcement reported more than 750,000 compatible checkpoints on the Hub.

How it works

Repositories

Every model, dataset or Space on the Hub is a Git repository. Large files such as weights are stored with large-file support. Each commit has a hash, so you can refer to an exact revision. Repositories can be public, private or gated, where users must accept terms before downloading.

Discovery

Users search the Hub by task, library, language, licence or size. The model card, usually the repository's README.md, explains what the model is for, how it was trained and evaluated, and its known limitations. Metadata in the card drives filters on the website. This idea is described further on the model cards page.

Loading a model

In Transformers, the from_pretrained method downloads configuration, weights and tokenizer files and builds the right Python class.

from transformers import pipeline

classifier = pipeline("sentiment-analysis",
                      model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")
print(classifier("The delivery was quick and the support team was helpful."))

Files are cached locally, so later runs do not download again.

You can fine-tune with the Transformers Trainer, with TRL for language model post-training, or with PEFT to train small adapter weights instead of the whole model. When you are done, push_to_hub uploads the model and a card to a new or existing repository, private or public.

Running models

Models can run locally, in a Space, or through hosted inference services. For high-throughput serving, teams often export Hub models to engines such as vLLM or llama.cpp, which read Hub formats directly.

Key concepts and terms

Term	Meaning
Hub	The platform hosting models, datasets and Spaces
Repository	A versioned Git store for one model, dataset or Space
Model card	Documentation of a model's use, data, evaluation and limits
Revision	A specific commit, branch or tag of a repository
Gated model	A repository that requires users to accept terms before access
Space	A hosted demo application
`from_pretrained`	Method that loads a model or tokenizer from the Hub
Pipeline	A high-level Transformers API for common tasks
Checkpoint	A saved set of model weights
safetensors	A safe, fast file format for tensors
Adapter	A small set of extra weights trained with PEFT
Access token	A credential used to read or write private repositories

A worked example

This scenario is illustrative. A team at a law office needs a model to sort incoming documents into eight categories, such as contracts, court filings and correspondence. They have 6,000 labelled documents and one GPU with 24 GB of memory.

They search the Hub for encoder models suited to classification, read the cards and check that the licence allows commercial use.
They pin a specific revision of the chosen base model so results are reproducible.
They put the labelled data into a Datasets object, split it 85 to 15 and tokenise it with the model's own tokenizer.
Using PEFT, they add LoRA adapters, so only a few per cent of the parameters are trained. Training fits comfortably in GPU memory.
They train with the Trainer for three epochs and measure macro-averaged F1 on the held-out set, looking closely at the smallest categories.
They push the adapter weights, the label mapping and a model card describing the data and its limits to a private repository in their organisation.
Colleagues load the base model plus adapter with two calls and run the classifier inside an internal tool.

The Hub becomes the team's shared record of which model, data and settings produced each result.

Strengths and limitations

Strengths:

A very large catalogue of open models and datasets in one place.
Standard interfaces that make switching models simple.
Version control and model cards that support reproducibility.
Libraries that cover the full path from loading to fine-tuning and sharing.
Private repositories and organisations for team use.

Limitations:

Quality and documentation vary widely between community uploads.
Licences differ from model to model and must be checked individually.
Downloaded code and older pickle-based weights can carry security risks.
Large models still need suitable hardware or a hosted service.
Fast-moving libraries sometimes change APIs between major versions.

How it compares with alternatives

Option	What it is	Best for
Hugging Face Hub	Open model and dataset hub with libraries	Finding, adapting and sharing open models
TorchVision and other domain libraries	Curated pretrained models inside a framework	Standard vision or audio baselines
KerasHub	Pretrained models for Keras	Teams working in Keras
MLflow model registry	Internal registry for a team's own models	Governance of in-house models
Cloud model gardens such as Vertex AI	Provider-hosted model catalogues	Managed deployment inside one cloud

Practical guidance and best practices

Read the model card and licence before using any model, and record both.
Pin a revision hash in production code rather than relying on the latest commit.
Prefer safetensors weights and avoid loading untrusted pickle files.
Be careful with trust_remote_code=True; it runs code from the repository on your machine.
Store access tokens in environment variables or secret stores, never in notebooks you share.
Use PEFT adapters when you need many fine-tuned variants of one base model.
Write a model card for every model you publish, including data sources and known limits.
Mirror important models internally if your service cannot depend on external downloads.

Common mistakes

Assuming that a public model may be used for any purpose without checking its licence.
Using a model's default revision in production, then seeing behaviour change after an upstream update.
Pairing a model with the wrong tokenizer, which silently ruins results.
Pushing private data or credentials to a public repository by mistake.
Choosing a model by downloads alone instead of testing it on your own data.
Ignoring the cache directory until it fills a disk.

Frequently asked questions

What is the Hugging Face Hub?

It is a platform for hosting and sharing machine learning models, datasets and demo applications. Each item lives in a versioned Git repository with documentation, and access can be public, private or gated.

Is Hugging Face free to use?

Public repositories and the open-source libraries are free. Paid plans and services add features such as more private storage, team management, hosted inference and dedicated hardware for Spaces.

Does Transformers still support TensorFlow?

Transformers version 5, announced in December 2025, moved to PyTorch as its single backend and phased out TensorFlow and Flax support. Older 4.x releases still include them.

What is safetensors and why does it matter?

It is a file format for model weights that stores only tensors and metadata. Unlike pickle files, loading it cannot run hidden code, and it loads quickly. It is the preferred format on the Hub.

Can I keep my models private?

Yes. Repositories can be private to you or your organisation, and gated repositories let owners control who may download. Sensitive work should still follow your organisation's data rules.

Both are open-source libraries maintained by Hugging Face. PEFT provides methods such as LoRA for cheap fine-tuning, and TRL provides post-training methods for language models. They work directly with Transformers models.

Glossary

Dataset card: documentation describing a dataset's contents, sources and limits.
Fine-tuning: further training a pretrained model on specific data.
Gradio: a Python library for building simple web demos, owned by Hugging Face.
Inference endpoint: a hosted service that runs a model behind an API.
LoRA: a method that trains small low-rank matrices instead of all weights.
Pickle: Python's serialisation format, which can execute code on load.
Tokenizer: the component that turns text into model input IDs.
Trust remote code: an option that allows custom code from a repository to run.

Summary

Hugging Face, founded in New York in 2016, grew from a chatbot company into the main hub of open machine learning. Its Hub stores models, datasets and Spaces as versioned repositories with model cards, and its libraries, led by Transformers, cover loading, fine-tuning and sharing. It makes open models easy to find and adapt, but users must still check licences, pin revisions, prefer safe weight formats and test every model on their own data.

It belongs to Frameworks in my stack, under Deep learning. Others in the same group:

PyTorch PyTorch Lightning TensorFlow Keras JAX