Python
print("Hello! 👋")- 1
- This is a simple Python command that prints the text “Hello” to the console. The output appears below the code block.
Hello! 👋To run or modify the example code yourself, you can open and download the Jupyter notebook underlying this page via the sidebar.
NoteQuestions about the code?
No problem! Explanations for the individual steps of the code are hidden behind the numbers on the right. Simply hover your mouse over the number to see the explanation. Where the respective code is executed (in Python, R, or the terminal) is shown in the header of the code block.
Load data
Regardless of the model used, we first load the already-installed packages as well as our text data, and store it in a list object:
Python
import pandas as pd
from transformers import pipeline
data = pd.read_csv("data/example_dataset.csv")
textlist = list(data["text"])- 1
- Importing the required libraries
- 2
- Loading the text data from a CSV file and storing it in a list
Load model
Unlike category-specific models, task-specific encoder models let us classify any category we like. As an example, we use the NLI model by Laurer et al. (2024) here.
TipGood to know
Instead of the model used here, you can also use other task-specific encoder models (usually labeled for “Zero-Shot-Classification”) for different languages and use cases. An overview of available models can be found on Hugging Face. Make sure that the chosen model is suitable for your specific task (e.g. sentiment analysis, topic classification, etc.) and supports the language of your texts.
Python
zeroshot_classifier = pipeline("zero-shot-classification",
model="MoritzLaurer/deberta-v3-base-zeroshot-v2.0")- 3
- Loading the task-specific encoder model for zero-shot classification
Classification
For classification with the NLI model, in addition to the model we also specify the category of interest as a candidate label (candidate_labels) and a “hypothesis” that will be checked against the text (hypothesis_template). When the pipeline() command runs, the category value is automatically inserted in place of the curly braces in the hypothesis. To run the classification, we apply the previously defined zeroshot_classifier command to our text data. We then extract the probability assigned to fear. We save the result in a DataFrame object for a clear overview.
Python
template = "This text expresses {}."
label = "fear"
results = zeroshot_classifier(textlist, candidate_labels=[label],
hypothesis_template=template)
result_df = pd.DataFrame({
"text": data["text"],
"fear_score": [r["scores"][0] for r in results],
})
print(result_df)- 4
- Defining the hypothesis with a placeholder for the category value
- 5
- Defining the category value to code for
- 6
- Applying the classification command to the text data with the specified candidate label and hypothesis
- 7
-
Assembling the original texts and their
fearprobabilities into a DataFrame for a clear overview - 8
- Printing the result DataFrame
References
Laurer, M., Atteveldt, W. van, Casas, A., & Welbers, K. (2024, March 22). Building efficient universal classifiers with natural language inference. https://doi.org/10.48550/arXiv.2312.17543