How to Build a Simple Generative AI application with Gradio

When you have a specific idea in mind, say summarizing an article, a small specialist model that is designed for that specific task can perform just as well as a General Purpose Large Language model. And interestingly, a smaller specialist model can also be cheaper and faster to run.

Build a Generative AI application with Gradio

You can, if you want, create an application that aggregates news articles from various sources and provides summarized versions of the articles for quick browsing. Or, you could develop a plugin that integrates with email services and automatically summarizes long emails, allowing users to quickly grasp the main points without reading the entire message.

But how would you do that? I'll show it in this brisk tutorial so that you can also give it a try. And the best part? It will not take hours but minutes.

Import libraries

import gradio as gr
from transformers import pipeline

Gradio is an open-source Python library, and you can accomplish a lot with it in minutes, like allowing users to input data, making predictions, and visualizing results with just a few lines of code.

And, we're importing the pipeline function from the Hugging Face Transformers library, which is very good for working with pre-trained transformer models in NLP.

Initialize a Summarization Pipeline

get_completion = pipeline("summarization", model="sshleifer/distilbart-cnn-12-6")

First off, we're going to use sshleifer/distilbart-cnn-12-6 model for text summarization because it is one of the state-of-the-art models known for its exceptional performance and accuracy in generating concise summaries. Another factor, if we use the Transformers Pipeline function for text summarization without specifying the model explicitly, it will still default to Distilbart CNN 12-6.

And most interestingly, the effective way to improve cost and speed is to create a smaller version of the model that has a very similar performance. This process, called distillation, is quite common. Distillation involves using the predictions of a large model to train a smaller one. The model we're using (Distilbart CNN 12-6) is actually a distilled version of the larger model trained by Facebook, known as the BART Large CNN model.

And since this model is built specifically for summarization, let's write some functions for any text that you feed into the model, so that it will output a summary of it.

Define Summarization Function

def summarize(input_text):
    # Generate the summary for the input text
    output = get_completion(input_text)
    # Extract and return the summary text
    return output[0]['summary_text']

Now we define a function summarize that takes input text as a parameter, generates a summary using the initialized summarization pipeline, and returns the summary text. This function simplifies usage and maintenance within the application.

Create Gradio Interface

gr.close_all()
demo = gr.Interface(
    fn=summarize, 
    inputs=[gr.Textbox(label="Text to summarize", lines=6)],
    outputs=[gr.Textbox(label="Result", lines=3)],
    title="Text Summarization with DistilBART-CNN",
    description="Summarize text using the `sshleifer/distilbart-cnn-12-6` model!"
)

Here, we set up the Gradio interface with input and output components, specify the summarization function, and provide a title and description to inform users about the interface's functionality.

Now, let's launch the interface so we can input text and receive summarized output using the DistilBART-CNN model.

demo.launch(share=True)

It will launch an interface like this.

Now that our application is up and running, the Gradio interface is accessible both locally and via the live link provisioned by Gradio. And we are now ready to test.

Unfortunately, there was a tragic train accident in Odisha, a state in India, last year. And if we summarize the text of this BBC article about it, we can see the output it gives.

Note: The model throws an error if your text exceeds 800 words. It's best to keep it between 700-800 words.

This works perfectly fine. You can do a lot of things if you have some experience in Python and use Gradio to build interfaces for your AI applications. Try this!

If you want to summarize books and papers, this tutorial by Raghavan Muthuregunathan about How to Summarize and Find Similar ArXiv Articles on Lablab.ai is a very helpful resource on the internet.

Credits

I’d be remiss not to give credit for this oversimplified tutorial to Apolinário Passos (Poli), a Machine Learning Art Engineer at Hugging Face. His short course on Deeplearning.ai with Andrew NG is an exhilarating resource for learning how to build AI-powered applications.

Good courses

• Generative AI Nanodegree
• AI Programming with Python
• Developing AI Applications with Python and Flask