Spaces:
Runtime error
Runtime error
| from deepsparse import Pipeline | |
| import time | |
| import gradio as gr | |
| markdownn = ''' | |
| # Sentiment Analysis Pipeline with DeepSparse | |
| Sentiment Analysis determines whether a certain sentence is positive or negative. | |
|  | |
| ### What is Deepsparse? | |
| DeepSparse is an inference runtime offering GPU-class performance on CPUs and APIs to integrate ML into your application. Sparsification is a powerful technique for optimizing models for inference, reducing the compute needed with a limited accuracy tradeoff. DeepSparse is designed to take advantage of model sparsity, enabling you to deploy models with the flexibility and scalability of software on commodity CPUs with the best-in-class performance of hardware accelerators, enabling you to standardize operations and reduce infrastructure costs. | |
| Similar to Hugging Face, DeepSparse provides off-the-shelf pipelines for computer vision and NLP that wrap the model with proper pre- and post-processing to run performantly on CPUs by using sparse models. | |
| SparseML Sentiment Analysis Pipelines integrate with Hugging Face’s Transformers library to enable the sparsification of a large set of transformers models. | |
| The Sentiment Analysis Pipeline has pre-trained models and pre-processing, enabling you to pass raw text to the model. | |
| ## Use Case Description | |
| An example use case for sentiment analysis is classifying the sentiment of customer reviews. | |
| Customer review classification is crucial for customer-facing enterprises across industries such as retail, entertainment, food, and beverage. Knowing what your customers say about your product or solution can help you quickly address negative customer reviews and in turn reduce churn, providing a better customer experience. | |
| #### Inference API Example | |
| Here is sample code for a sentiment analysis pipeline: | |
| ```python | |
| from deepsparse import Pipeline | |
| pipeline = Pipeline.create(task="sentiment-analysis", model_path="zoo:nlp/sentiment_analysis/bert-base/pytorch/huggingface/sst2/12layer_pruned80_quant-none-vnni") | |
| text = "kept wishing I was watching a documentary about the wartime Navajos and what they accomplished instead of all this specious Hollywood hoo-ha" | |
| inference = pipeline(text) | |
| print(inference) | |
| ``` | |
| [Want to train a sparse model on your data? Checkout the documentation on sparse transfer learning](https://docs.neuralmagic.com/use-cases/natural-language-processing/text-classification) | |
| ''' | |
| task = "sentiment-analysis" | |
| sparse_qa_pipeline = Pipeline.create( | |
| task=task, | |
| model_path="zoo:distilbert-sst2_wikipedia_bookcorpus-pruned90", | |
| top_k = 2 | |
| ) | |
| def run_pipeline(text): | |
| sparse_start = time.perf_counter() | |
| sparse_output = sparse_qa_pipeline(text) | |
| sparse_result = dict(sparse_output) | |
| sparse_end = time.perf_counter() | |
| sparse_duration = (sparse_end - sparse_start) * 1000.0 | |
| dict_r = {"Negative":sparse_result['scores'][0][0],"Positive": sparse_result['scores'][0][1] } | |
| return dict_r, sparse_duration | |
| with gr.Blocks() as demo: | |
| with gr.Row(): | |
| with gr.Column(): | |
| gr.Markdown(markdownn) | |
| with gr.Column(): | |
| gr.Markdown(""" | |
| ### Sentiment analysis demo | |
| Using [huggingface/sst2](https://sparsezoo.neuralmagic.com/models/nlp%2Fsentiment_analysis%2Fbert-base%2Fpytorch%2Fhuggingface%2Fsst2%2F12layer_pruned80_quant-none-vnni) | |
| """) | |
| text = gr.Text(label="Text") | |
| btn = gr.Button("Submit") | |
| sparse_answers = gr.Label(label="Sparse model answers",num_top_classes=2) | |
| sparse_duration = gr.Number(label="Sparse Latency (ms):") | |
| gr.Examples([["kept wishing I was watching a documentary about the wartime Navajos and what they accomplished instead of all this specious Hollywood hoo-ha ."],["Kidman is really the only thing that 's worth watching in Birthday Girl , a film by the stage-trained Jez Butterworth -LRB- Mojo -RRB- that serves as yet another example of the sad decline of British comedies in the post-Full Monty world .","One of the other reviewers has mentioned that after watching just 1 Oz episode you'll be hooked. They are right, as this is exactly what happened with me.<br /><br />The first thing that struck me about Oz was its brutality and unflinching scenes of violence, which set in right from the word GO. Trust me, this is not a show for the faint hearted or timid. This show pulls no punches with regards to drugs, sex or violence. Its is hardcore, in the classic use of the word.<br /><br />It is called OZ as that is the nickname given to the Oswald Maximum Security State Penitentary. It focuses mainly on Emerald City, an experimental section of the prison where all the cells have glass fronts and face inwards, so privacy is not high on the agenda. Em City is home to many..Aryans, Muslims, gangstas, Latinos, Christians, Italians, Irish and more....so scuffles, death stares, dodgy dealings and shady agreements are never far away.<br /><br />I would say the main appeal of the show is due to the fact that it goes where other shows wouldn't dare. Forget pretty pictures painted for mainstream audiences, forget charm, forget romance...OZ doesn't mess around. The first episode I ever saw struck me as so nasty it was surreal, I couldn't say I was ready for it, but as I watched more, I developed a taste for Oz, and got accustomed to the high levels of graphic violence. Not just violence, but injustice (crooked guards who'll be sold out for a nickel, inmates who'll kill on order and get away with it, well mannered, middle class inmates being turned into prison bitches due to their lack of street skills or prison experience) Watching Oz, you may become comfortable with what is uncomfortable viewing....thats if you can get in touch with your darker side."],["Phil the Alien is one of those quirky films where the humour is based around the oddness of everything rather than actual punchlines.<br /><br />At first it was very odd and pretty funny but as the movie progressed I didn't find the jokes or oddness funny anymore.<br /><br />Its a low budget film (thats never a problem in itself), there were some pretty interesting characters, but eventually I just lost interest.<br /><br />I imagine this film would appeal to a stoner who is currently partaking.<br /><br />For something similar but better try Brother from another planet"]],inputs=[text],) | |
| btn.click( | |
| run_pipeline, | |
| inputs=[text], | |
| outputs=[sparse_answers,sparse_duration], | |
| ) | |
| if __name__ == "__main__": | |
| demo.launch() | |