Back to Article
Classification with a generative decoder
Download Notebook

Classification with a generative decoder

This notebook belongs to the DGPs 2026 workshop Text Classification with Open Source Models.

You’ll classify the same open-ended survey text from the Real World Worry Waves Dataset (RW3D) as in the previous notebook — this time with a small, open-source generative decoder model.

Instead of a fixed set of labels, you write a prompt, the model generates free text, and you parse that text back into a structured answer.

How to use this notebook: run each cell from top to bottom with Shift+Enter or by clicking the run button. Most cells are given and fully explained; a few are marked Your turn — those are where you edit or write something yourself. Every exercise cell has a safe default, so nothing breaks if you leave it as-is, but you’ll get more out of the session if you actually change it.

Important note for Colab users: If you are running this notebook in Google Colab, please follow these steps: 1. Make sure you have a Google account and are logged in. 2. To create a copy of this notebook in your own Google Drive (advised), select “Copy to Drive” at the top of the Colab page. 3. Once the notebook copy opens in Colab, click on “Runtime” in the menu bar, then select “Change runtime type” and choose “T4 GPU” as the hardware accelerator. 4. Click “Save” to apply the changes. 5. Now you can run the notebook cells as instructed.

Setup

First, we install the Python packages we need. This will take some time to complete.

In [3]:
%pip install -q transformers accelerate pandas

Next, we can change the logging level of the transformers library to avoid cluttering the notebook with warnings and info messages.

You do not have to run this cell, but it will make the output easier to read.

In [1]:
from transformers import logging
logging.set_verbosity_error()

Code-Along

We’ll build the classification workflow together.

There are a couple of small “your turn” fill-ins along the way that you can work on after the code-along.

Step 1: Load the data

We’ll use the same curated sample of survey responses from RW3D as in the previous notebook.

First, let’s load the data. Because generative decoder models are slower than encoder models, we’ll only use a small sample of 10 texts for this notebook. You can change the number of rows to load by changing the argument to head() in the next cell.

In [2]:
import pandas as pd

DATA_URL = "https://raw.githubusercontent.com/felixdidi/llm-content-analysis/main/sections/workshop/data/rw3d_workshop_sample.csv"
df = pd.read_csv(DATA_URL).head(10)
df
id text main_emotion anger fear sadness
0 1 Stressed and uninformed. Don't feel enough is ... anger 9 1 6
1 2 I feel worried about the virus, burnout about ... fear 8 9 7
2 3 Disgusted how unprotected NHS staff are and ho... sadness 2 6 7
3 4 I am not too worried about the situation as I ... sadness 1 3 6
4 5 I'm finding the Corona situation quite dauntin... sadness 5 2 7
5 6 At the moment I feel like I am doing everythin... sadness 1 2 2
6 7 I feel sad for all the people infected and tho... sadness 2 3 4
7 8 I'm worried about elderly parents. I'm worried... anger 9 7 8
8 9 I am very anxious and worried about the COVID-... fear 2 9 9
9 10 i'm very Angry at the moment as there are many... anger 8 8 6

Step 2: Load a small open-source decoder model

We’ll use Qwen2.5-1.5B-Instruct, an instruction-tuned model small enough to run comfortably on Colab’s free GPU (and likely also your own laptop, if you intend to run it locally).

Unlike the encoder models from the previous notebook, this one wasn’t trained to return scores for a fixed set of labels. Instead, it generates a stream of text, so we’ll need to tell it what we want in the prompt itself, and turn its answer back into something structured ourselves.

Generative decoder models are also much larger than encoder models. Loading the model may take some time.

In [3]:
from transformers import pipeline

generate = pipeline(
    "text-generation",
    model="Qwen/Qwen2.5-1.5B-Instruct",
    device_map="auto",
)

We write a short system prompt that states the task and asks for a strict JSON answer — here, whether the text expresses fear (true/false) plus a short reasoning. Because the prompt doesn’t show the model any examples, classifications will be based only on the context knowledge of the model. This is called zero-shot prompting.

Unlike the encoder pipelines from the previous notebook, a decoder’s generate() call doesn’t batch cleanly over a list of texts out of the box — so we call it once per text, in a for loop. do_sample=False keeps answers deterministic, so re-running the cell gives the same result. Running it on all texts may take some time.

In [4]:
import json

results = []
for text in df['text']:
    instructions = [
        {"role": "system",
         "content": "You are a trained assistant for content analysis who determines whether a text expresses fear. Always answer precisely in JSON format with fear (true or false) and reasoning (a short justification in English). Return only the JSON response, starting with '{' and ending with '}', with these two parameters."},
        {"role": "user", "content": text}]
    outputs = generate(instructions, max_new_tokens=256, do_sample=False)
    results.append({"text": text, "response": outputs[0]["generated_text"][-1]['content']})

results
[{'text': "Stressed and uninformed. Don't feel enough is being done by the government to help with the NHS crisis or the proper enforcement of social distancing.nhs staff should be tested immediately and given the right equipment to fight the Corona virus. There should be a full lockdown straight away to stop the spread.the equipment side of it is the fact that they aren't really getting enough or even any at all.how can you be expected to take urgent care of someone, and also be worried that you might catch it,or maybe even already have it.",
  'response': '```json\n{"fear": true, "reasoning": "The text expresses significant concern about the current situation regarding the NHS crisis, lack of action from the government, inadequate testing and equipment for healthcare workers, and the need for immediate measures such as a full lockdown to control the spread of the coronavirus. This indicates a high level of anxiety and fear related to public health issues."}\n```'},
 {'text': 'I feel worried about the virus, burnout about me getting it. My mother is undergoing chemotherapy and has been told no one knows if treatment will continue if nhs becomes overwhelmed. So I am worried most about her treatment ending  also, because of the chemotherapy her immune system is lower and it could really hurt her if she caught it. So at the moment I am just keeping up to date on numbers and nhs capacity and taking it one chemotherapy session at a time. Also, my partner has underlying health conditions and is self isolating in his house. I havent seen him for nearly a month and I am worried about when I will get to see him with all this going on',
  'response': '```json\n{\n  "fear": true,\n  "reasoning": "The text expresses significant worry about multiple aspects including personal health, family members\' health, and overall societal impact due to the pandemic."\n}\n```'},
 {'text': 'Disgusted how unprotected NHS staff are and how late the government has been in trying to resolve this. Disgusted at the amount of people not adhering to guidelines for the safety of others. Sad about the amount of people dying without their families, dying alone without even care staff having the time to sit and hold their hand whilst they take their last breath. Disgusted at the attitude of current staff when someone needs to take the week off, but also feeling the pressure of having to stay at work even though they are experiencing symptoms themselves.',
  'response': '```json\n{"fear": true, "reasoning": "The text expresses strong negative emotions such as disgust, sadness, and anger towards various aspects including healthcare workers, public adherence to guidelines, deaths due to lack of support, and staff attitudes during illness."}\n```'},
 {'text': 'I am not too worried about the situation as I believe I am a fit and healthy human being. I am more worried about family members becoming ill with the disease and I am also very sad that I cannot see them at this time. Thankfully with technology these days I can see them everyday through a screen and chat to both friends and family whenever I want to. Its just annoying that I cannot leave the house apart from to get essentials, I live in a flat and with the nice weather we are having its becoming quite a struggle.',
  'response': '```json\n{\n  "fear": false,\n  "reasoning": "The text does not express any significant level of fear related to the situation described. The author is primarily concerned with their own health and well-being, expressing sadness over the inability to visit family members due to restrictions caused by the disease."\n}\n```'},
 {'text': "I'm finding the Corona situation quite daunting and draining. I can't wait for it to end, and for things to get back to normal. I feel things will not be the same again.it's left so many families devastated. I feel so sad for the families that have lost loved ones,  both old and young. For those that are fighting for their lives, I hope they pull through.  I have hope that all will be well in the end.  This shall pass, we shall overcome this by God's grace. We are all in this together as this situation has affected the whole world and not just certain parts of it. I salute all those working in the frontline, the NHS and all those volunteering at this time.",
  'response': '```json\n{\n  "fear": false,\n  "reasoning": "The text expresses hope, resilience, and solidarity during the pandemic, indicating a lack of fear."\n}\n```'},
 {'text': 'At the moment I feel like I am doing everything I can to protect myself as an individual from the Covid-19 virus. The UK government were a little slow to react and did not take it as seriously as I would have liked, but seem to be doing and saying the right things now. I am looking at other countries data from around the world and things are starting to look more positive from Italy and Spain, where hopefully the figures have peaked as they continue to ‘flatten the curve’. In America the state of New York they still seem to be having a real problem with getting the situation under control, I hope the president will listen to health experts as they monitor the situation.',
  'response': '```json\n{\n  "fear": false,\n  "reasoning": "The text does not express any significant level of fear related to the coronavirus pandemic. Instead, it focuses on personal protective measures, governmental responses, and optimistic outlooks regarding global trends."\n}\n```'},
 {'text': 'I feel sad for all the people infected and those that died and their families. Somewhat pissed off at how the government have handled the outbreak airports and eurotunnel still open...Particulary concerned at Boris Johnson with the situation of him still continuing to shake hands even with coronavirus patients...no wonder he is in hospital at the moment showas a disregard for the situation, fear the worse is yet to come. Me personally I am not worried health wise more concerned for my eldely parents and of course the effect it will have on our everyday lives moving forward',
  'response': '```json\n{"fear": true, "reasoning": "The text expresses concern about the pandemic\'s impact on individuals, families, and society as a whole, indicating a level of worry and fear regarding the situation."}\n```'},
 {'text': "I'm worried about elderly parents. I'm worried about my kids education. I feel exhausted and overwhelmed. I worry about my kids getting infected. I starting to worry less about food and provisions tho we still have no toilet paper. I feel disappointed for my kids and for myself on fun things we can't do. I feel like im doing my bit but Im so upset about not seeing my mum and mother in law who are on their own  every day feels the same and Im starting to get bored too.\nThe kids are acting up and fighting. I feel like my husband doesnt have to deal with this as much. Im trying to work too. ",
  'response': '```json\n{\n  "fear": true,\n  "reasoning": "The text expresses multiple concerns including worries about health, finances, family dynamics, and personal stress related to daily life challenges."\n}\n```'},
 {'text': 'I am very anxious and worried about the COVID-19 Coronavirus. I am scared I might catch it, and that my family and friends will catch it. I am scared we will die. I am scared of the daily numbers. I am scared to leave the house. I am fearful of what the future holds, and what will be left standing when we leave the house. I miss my family and friends. I miss being able to leave the house to go into town for coffee, to eat out, to socialise, to shop etc... I am anxious when I see other people and not to go too close to them.',
  'response': '```json\n{\n  "fear": true,\n  "reasoning": "The text expresses significant anxiety and worry about the coronavirus pandemic, including fears related to catching the virus, losing loved ones, missing social activities, and uncertainty about the future."\n}\n```'},
 {'text': "i'm very Angry at the moment as there are many people who are not taking the situation seriously and are still going out of the house for unnecessary reasons. i have been isolating for the past 2 weeks due to being high risk and the more people dont listen the longer i have to stay home and not see my family and friends. My parents and inlaws are missing my daughter and all her milestones and being pregnant right now is stressful as it is.\nmy mother is also high risk as she is 73 and has memory issues so has still been going out so it stresses me out having to worry about her also. ",
  'response': '```json\n{\n  "fear": true,\n  "reasoning": "The text expresses concern and anxiety about the current situation, including isolation, lack of social interaction, stress related to health risks, and worries about vulnerable individuals like the narrator\'s mother. The use of strong emotional language such as\'very Angry\' indicates heightened fear."\n}\n```'}]

Basically, that was it. We now have a response for every text in the sample. results is just a list of dictionaries — one per text, each holding the original text and the model’s raw response string — and that’s exactly the structure of a JSON file. So, you could simply save it with Python’s built-in json module:

with open("results.json", "w") as f:
    json.dump(results, f)

That file could be picked up in R or any other tool from here. We’ll keep working in Python below.

If we want to keep working in Python, we need to parse the model’s free-text output back into a structured answer. We use some simple string manipulation to extract the fear value and the reasoning text. This is a bit fragile because the model’s output may not always be perfectly formatted. For example, we can see that, although we told the model to return a strict JSON answer beginning with {, the model consistently returns the JSON object wrapped in a markdown code block (```json\n{...}\n```). We can strip that away with some simple string manipulation, which is already implemented in the parse_response() function below.

In [5]:
def parse_response(response):
    response = response.strip().strip("`").removeprefix("json").strip()
    try:
        parsed = json.loads(response)
        return parsed['fear'], parsed['reasoning']
    except (json.JSONDecodeError, KeyError):
        return None, None

parsed_data = [(entry['text'], *parse_response(entry['response'])) for entry in results]
parsed_data = pd.DataFrame(parsed_data, columns=['text', 'fear', 'reasoning'])
parsed_data
text fear reasoning
0 Stressed and uninformed. Don't feel enough is ... True The text expresses significant concern about t...
1 I feel worried about the virus, burnout about ... True The text expresses significant worry about mul...
2 Disgusted how unprotected NHS staff are and ho... True The text expresses strong negative emotions su...
3 I am not too worried about the situation as I ... False The text does not express any significant leve...
4 I'm finding the Corona situation quite dauntin... False The text expresses hope, resilience, and solid...
5 At the moment I feel like I am doing everythin... False The text does not express any significant leve...
6 I feel sad for all the people infected and tho... True The text expresses concern about the pandemic'...
7 I'm worried about elderly parents. I'm worried... True The text expresses multiple concerns including...
8 I am very anxious and worried about the COVID-... True The text expresses significant anxiety and wor...
9 i'm very Angry at the moment as there are many... True The text expresses concern and anxiety about t...

Your turn

The prompt above only asks about fear. Edit the system prompt so it asks for anger, fear, and sadness all at once (three booleans plus one reasoning, in a single JSON object). If everything works correctly, the code below should pull out all three fields instead of just fear. Re-run the cell — how many responses still parse cleanly?

Feeling bold? If parsing below fails for some responses, you can try to improve the parsing function to handle more edge cases. Look at the raw output to see what the model is returning, and try to make the parsing more robust. You can also try to improve the prompt itself to get cleaner output from the model or increase the maximum number of tokens in the generate() call if the model is truncating its output.

In [12]:
results_all_three = []
for text in df['text']:
    instructions = [
        {"role": "system",
         "content": "ADD YOUR UPDATED SYSTEM PROMPT HERE"}, # TODO
        {"role": "user", "content": text}]
    outputs = generate(instructions, max_new_tokens=256, do_sample=False)
    results_all_three.append({"text": text, "response": outputs[0]["generated_text"][-1]['content']})

results_all_three
In [13]:
def parse_response_all_three(response):
    response = response.strip().strip("`").removeprefix("json").strip()
    try:
        parsed = json.loads(response)
        return parsed['fear'], parsed['anger'], parsed['sadness'], parsed['reasoning']
    except (json.JSONDecodeError, KeyError):
        return None, None, None, None

parsed_data_all_three = [(entry['text'], *parse_response_all_three(entry['response'])) for entry in results_all_three]
parsed_data_all_three = pd.DataFrame(parsed_data_all_three, columns=['text', 'fear', 'anger', 'sadness', 'reasoning'])
parsed_data_all_three

Exercise: From zero-shot to few-shot prompting

So far, the model has only ever seen the text it needs to classify — no examples of what a “correct” answer looks like. That’s zero-shot prompting. In few-shot prompting, you show the model one or more examples (e.g., an example text, maybe the JSON answer you’d want for it) before asking about the real text. This can nudge the model toward your specific understanding of the category and toward the exact output format you asked for. However, it may also bias the model toward the examples you show it, so you have to be careful about what examples you choose.

Below, write 1–2 short example texts, together with the JSON answer you think is correct for each — base this on your own understanding of what should count as fear here. These examples will be inserted into the prompt before the real text every time the model is called.

In [15]:
# TODO: write 1-2 short examples of your own: a text, and the JSON answer
# you think is correct for it (based on your own understanding of "fear").
# These will be shown to the model as worked examples before it classifies
# the real texts below.
few_shot_examples = [
    {
        "text": "EXAMPLE TEXT 1",
        "response": 'EXAMPLE RESPONSE 1',
    },
    {
        "text": "EXAMPLE TEXT 2",
        "response": 'EXAMPLE RESPONSE 2',
    },
]

Now we run the same loop as above, but this time we add your examples as extra turns in the conversation before the real text — each example becomes one user turn (the example text) followed by one assistant turn (the example answer), exactly how it would look if the model had already answered it.

In [17]:
results_fewshot = []
for text in df['text']:
    instructions = [
        {"role": "system",
         "content": "You are a trained assistant for content analysis who determines whether a text expresses fear. Always answer precisely in JSON format with fear (true or false) and reasoning (a short justification in English). Return only the JSON response, starting with '{' and ending with '}', with these two parameters."},
    ]
    for example in few_shot_examples:
        instructions.append({"role": "user", "content": example["text"]})
        instructions.append({"role": "assistant", "content": example["response"]})
    instructions.append({"role": "user", "content": text})

    outputs = generate(instructions, max_new_tokens=256, do_sample=False)
    results_fewshot.append({"text": text, "response": outputs[0]["generated_text"][-1]['content']})

results_fewshot

Parse the few-shot responses the same way as before, and compare them to the zero-shot predictions from the code-along side by side.

In [19]:
parsed_data_fewshot = [(entry['text'], *parse_response(entry['response'])) for entry in results_fewshot]
parsed_data_fewshot = pd.DataFrame(parsed_data_fewshot, columns=['text', 'fear', 'reasoning'])

comparison = parsed_data[['text', 'fear']].merge(
    parsed_data_fewshot[['text', 'fear']],
    on='text', suffixes=('_zero_shot', '_few_shot'),
)
comparison

Look at where the two columns disagree. Few-shot prompting can make a model’s answers more consistent with your specific definition of a category, and can improve how reliably it sticks to the requested format — but it isn’t free. The examples you write inevitably steer the model’s judgment (a form of researcher bias that’s easy to underestimate with just 1–2 examples), each call now sends a longer prompt through the model (this loop was already slow; more examples means more tokens, means more time), and there’s no guarantee the model is actually applying your examples’ reasoning to a new, genuinely ambiguous case rather than just imitating their tone or length.

Bonus: does a smaller model still hold up?

If you have time left, try the (zero-shot) classification from the code-along again, but with an even smaller model this time: Qwen2.5-0.5B-Instruct, a third of the size of the one we’ve used so far. Does it still reliably return valid JSON, or do more responses fail to parse than with the 1.5B model? Where it does parse, do the label decisions still look reasonable to you?

In [21]:
alternative_generator = pipeline(
    "text-generation",
    model="Qwen/Qwen2.5-0.5B-Instruct",
    device_map="auto",
)
In [22]:
# TODO: reuse the loop from Step 3 (swap generate for alternative_generator) to
# classify df['text'] again, then parse the responses with parse_response and
# check how well the smaller model performs compared to the larger one.