🌑

Stephen's Blog

Home Archives About

Hugging Face Cheat Sheet

Stephen Cheng

 

Intro

The Hugging Face is a Git-backed platform for discovering, running, training, and sharing machine learning models and datasets, with a strong emphasis on open-source and reproducibility. The core libraries are: transformers (models + pipelines), tokenizers (fast tokenization), datasets (data loading/processing), and huggingface_hub (Hub interaction + versioning).

  • A model is a pretrained checkpoint.
  • A tokenizer converts raw text into tokens.
  • A pipeline bundles preprocessing, inference, and postprocessing for a task.
  • A dataset is an Arrow-backed collection of data with splits (train/validation/test).
  • A checkpoint is a saved snapshot of model weights/config.
  • A inference means running a trained model on new inputs.
  • A repo is a Git-backed Hub unit storing models/datasets/Spaces.

Transformers

  • Sentiment analysis
1
2
3
4
5
6
7
8
9
10
11
12
from transformers import pipeline

# Create a pipeline by specifying a task and model ID
analyze_sentiment = pipeline(
   "sentiment-analysis",
   model="distilbert-base-uncased-finetuned-sst-2-english"
)

# Run inference on input text
result = analyze_sentiment("Hugging Face makes NLP workflows not easy!")

print(result)
  • Text summarization
1
2
3
4
5
6
7
8
9
10
from transformers import pipeline

# Create a summarization pipeline
summarize_text = pipeline(
   "summarization",
   model="facebook/bart-large-cnn"
)

# Summarize input text
summarize_text("Long document text goes here...")
  • Document question answering
1
2
3
4
5
6
7
8
9
10
11
12
13
from transformers import pipeline

# Create a document QA pipeline
answer_question = pipeline(
   "document-question-answering",
   model="impira/layoutlm-document-qa"
)

# Ask a question about a document image
answer_question(
   image="invoice.png",
   question="What is the invoice total?"
)
  • Inference manually
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_id = "distilbert-base-uncased-finetuned-sst-2-english"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

inputs = tokenizer("Hugging Face is not great", return_tensors="pt")

with torch.no_grad():
   outputs = model(**inputs)

result = outputs.logits.argmax(dim=-1).item()

print(result)

Datasets

  • Load and slice datasets
1
2
3
4
5
6
7
8
from datasets import load_dataset

movie_reviews = load_dataset("stanfordnlp/imdb")

train_reviews = movie_reviews["train"]
train_reviews[0]

small_sample = train_reviews.select(range(100))
  • Preprocess a dataset
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
from datasets import load_dataset
from transformers import AutoTokenizer

dataset = load_dataset("stanfordnlp/imdb")
tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")

def tokenize_batch(batch):
   return tokenizer(
       batch["text"],
       truncation=True,
       padding="max_length",
       max_length=256
   )

tokenized_dataset = dataset.map(
   tokenize_batch,
   batched=True,
   remove_columns=["text"]
)

Working with huggingface_hub

  • Save locally and reload
1
2
3
4
5
6
7
8
9
10
11
12
from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_id = "distilbert-base-uncased-finetuned-sst-2-english"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

tokenizer.save_pretrained("local_tokenizer")
model.save_pretrained("local_model")

AutoTokenizer.from_pretrained("local_tokenizer")
AutoModelForSequenceClassification.from_pretrained("local_model")
  • Login to the Hub
1
2
3
from huggingface_hub import login

login()
  • Upload a model to the Hub
1
2
3
4
5
6
7
8
9
from transformers import AutoTokenizer, AutoModelForSequenceClassification

repo_id = "username/model"

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")
model = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased")

tokenizer.push_to_hub(repo_id)
model.push_to_hub(repo_id)

, , — Oct 11, 2025

Search

    Made with ❤️ and ☀️ on Earth.

    GitHub LinkedIn Instagram