A Hands-On Guide to AI Text Generation
Run and Deploy a Free Text Generator in Python using GPT-2 and Streamlit
How exactly does AI generate coherent and creative sentences? In this article, we’ll walk through a simple yet powerful Python script that uses OpenAI’s GPT-2 model to generate text. By the end, you’ll understand how language models like GPT-2 work, and you’ll be able to experiment with AI-generated storytelling yourself.
What Is GPT-2?
GPT-2 (Generative Pre-trained Transformer 2) is a Deep Learning model developed by OpenAI that generates human-like text. It is based on the Transformer architecture, which revolutionized NLP by enabling machines to process context and structure in a previously impossible way.
Though it’s a bit older and less advanced than newer models like GPT-4 or beyond, it’s lightweight, open-source, and can run on modest hardware, making it a fantastic choice for creative projects, including:
- Creative writing — Generating unique, surprising pieces of text.
- Chatbots and virtual assistants — Powering AI-driven conversations.
- Code completion — Assisting developers by predicting the following lines of code.
Let’s break down a simple Python script that generates text using GPT-2.
Step 1: Importing the Required Libraries
To use GPT-2, we need the transformers library, which provides pre-trained models for NLP tasks:
pip install transformers torchNow, let’s import the necessary components:
from transformers import GPT2LMHeadModel, GPT2TokenizerGPT2LMHeadModel: The actual GPT-2 model that generates text.GPT2Tokenizer: Converts text into numerical tokens that the model can understand.
By loading these components, we can interact with the GPT-2 model in just a few lines of code.
Step 2: Loading the Pre-Trained Model
GPT-2 is available in multiple sizes (gpt2, gpt2-medium, gpt2-large, and gpt2-xl). Here, we use the smallest version (gpt2) for efficiency:
model_name = "gpt2"
model = GPT2LMHeadModel.from_pretrained(model_name)
tokenizer = GPT2Tokenizer.from_pretrained(model_name)This loads the model and tokenizer, allowing us to convert text into a format that GPT-2 can process.
Step 3: Encoding the Input Text
To generate text, GPT-2 needs a starting prompt. In our case, we’ll use the classic phrase:
input_text = "Once upon a time"
input_ids = tokenizer.encode(input_text, return_tensors="pt")Here’s what’s happening:
✅ Tokenization — The input text is converted into numerical representations (tokens).
✅ Return tensors — The return_tensors="pt" argument ensures that the output is compatible with PyTorch (the Deep Learning framework used by GPT-2).
At this point, GPT-2 is ready to generate text!
Step 4: Generating AI-Powered Text
Now, we use the .generate() function with custom parameters to fine-tune the output:
output = model.generate(
input_ids,
max_length=50, # Maximum length of the generated text
num_return_sequences=1, # Generate one output sequence
temperature=0.7, # Controls randomness (higher = more creative)
top_k=50, # Limits to top 50 probable words
top_p=0.9, # Nucleus sampling (focuses on high-probability words)
repetition_penalty=1.2, # Reduces repetition
do_sample=True # Enables randomness in generation
)Let’s break down these parameters:
max_length=50– Ensures the generated text doesn’t exceed 50 tokens.temperature=0.7– Adjusts randomness. Lower values make the output more predictable, while higher values make it more creative.top_k=50– Limits the model to selecting from the top 50 most likely words.top_p=0.9– Ensures that only high-probability words are considered.repetition_penalty=1.2– Discourages repetitive text.do_sample=True– Enables randomness instead of always picking the most probable word.
These settings help balance creativity and coherence.
Step 5: Decoding the Generated Text
After GPT-2 generates text, we need to convert the numerical tokens back into readable text:
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated_text)The skip_special_tokens=True ensures that unnecessary tokens (like end-of-sequence markers) don’t appear in the final output.
The Result: AI-Generated Storytelling
Running this script produces different results each time, but here’s an example output:
Input:
🚀 “Once upon a time”
Output:
📝 “Once upon a time, in a faraway kingdom, a young prince discovered a hidden portal to a world beyond imagination. He stepped inside, unaware that his journey would change the fate of the kingdom forever.”
Not bad for an AI, right?
Why This Matters
GPT-2 is a powerful tool with real-world applications. Whether content creation, chatbots, or creative writing assistance, AI-driven text generation is transforming industries.
However, AI-generated text also comes with challenges:
- Bias & Ethics — Language models can unintentionally reflect biases from their training data.
- Misinformation — AI can generate convincing but false narratives.
- Computational Cost — Running large models requires significant processing power.
Turn This Into an Interactive Streamlit App
What if you could turn this text generator into an interactive web app without writing a single line of front-end code?
That’s precisely what Streamlit allows you to do. Find next the code to run a user-friendly AI-powered text generator in your browser.
import streamlit as st
from transformers import GPT2LMHeadModel, GPT2Tokenizer
# Load pre-trained model and tokenizer
model_name = "gpt2"
model = GPT2LMHeadModel.from_pretrained(model_name)
tokenizer = GPT2Tokenizer.from_pretrained(model_name)
# Streamlit app
st.title("Text Generation with GPT2")
# Input text
input_text = st.text_area("Enter your prompt:", "Once upon a time")
# Parameters in the sidebar
st.sidebar.title("Tune the Parameters")
max_length = st.sidebar.slider(
"Max Length",
min_value=10,
max_value=200,
value=50,
help="The maximum length of the generated text, including the input text."
)
temperature = st.sidebar.slider(
"Temperature",
min_value=0.1,
max_value=1.0,
value=0.7,
help="Controls the randomness of predictions by scaling the logits before applying softmax. Lower values make the model more confident and deterministic, while higher values increase randomness."
)
top_k = st.sidebar.slider(
"Top K",
min_value=0,
max_value=100,
value=50,
help="Limits the sampling pool to the top K tokens with the highest probabilities. This helps in reducing the likelihood of generating less probable tokens."
)
top_p = st.sidebar.slider(
"Top P",
min_value=0.0,
max_value=1.0,
value=0.9,
help="Limits the sampling pool to the smallest set of tokens whose cumulative probability is greater than or equal to P. This helps in balancing between deterministic and random sampling."
)
repetition_penalty = st.sidebar.slider(
"Repetition Penalty",
min_value=1.0,
max_value=2.0,
value=1.2,
help="Penalizes repeated tokens to reduce the likelihood of repetitive text generation."
)
num_return_sequences = st.sidebar.slider(
"Number of Sequences",
min_value=1,
max_value=5,
value=1,
help="The number of different sequences to generate."
)
# Generate text button
if st.button("Generate Text"):
# Encode input text
input_ids = tokenizer.encode(input_text, return_tensors="pt")
# Generate text with fine-tuned parameters
output = model.generate(
input_ids,
max_length=max_length,
num_return_sequences=num_return_sequences,
temperature=temperature,
top_k=top_k,
top_p=top_p,
repetition_penalty=repetition_penalty,
do_sample=True
)
# Decode and display the generated text
for i in range(num_return_sequences):
generated_text = tokenizer.decode(output[i], skip_special_tokens=True)
st.write(generated_text)How does this work?
✅ User Input: You enter a text prompt in the provided text box.
✅ Adjustable Parameters: You can modify max_length, temperature, top_k, top_p, and repetition_penalty.
✅ AI-Generated Text: When you click "Generate Text", GPT-2 processes your input and generates a unique response.
✅ Interactive & Real-Time: The app provides instant feedback and allows experimentation with different AI settings.
Run the App in your CMD with: streamlit run [file_name].py
Try It Yourself!
Want to experiment with AI-generated text? Copy and run the script in your own Python environment. You can tweak parameters like temperature and max_length to see how they affect the output.
Try fine-tuning GPT-2 on custom datasets for a more powerful text generation, or explore OpenAI’s latest models.
The whole code in this article is available in the following GitHub repo.
