Word clouds are a type of data visualization in which various words from a dataset are actuated. Words that are larger in the word cloud are more common and words in the middle are also more common. In addition, some word clouds even use various colors to indicated importance.
This post will provide an example of how to make a word cloud using python. We will be using the “Women’s E-Commerce Clothing Reviews” available on the kaggle website. We are going to only use the text reviews to make our word cloud even though other data is in the dataset. To prepare our dataset for making the word cloud we need to the following.
- Lowercase all words
- Remove punctuation
- Remove stopwords
After completing these steps we can make the word cloud. First, we need to load all of the necessary modules.
import pandas as pd import re from nltk.corpus import stopwords import wordcloud import matplotlib.pyplot as plt
We now need to load our dataset we will store it as the object ‘df’
df=pd.read_csv('YOUR LOCATION HERE')
df.head()
It’s hard to read but we will be working only with the “Review Text” column as this has the text data we need. Here is what our column looks like up close.
df['Review Text'].head() Out[244]: 0 Absolutely wonderful - silky and sexy and comf... 1 Love this dress! it's sooo pretty. i happene... 2 I had such high hopes for this dress and reall... 3 I love, love, love this jumpsuit. it's fun, fl... 4 This shirt is very flattering to all due to th... Name: Review Text, dtype: object
We will now make all words lower case and remove punctuation with the code below.
df["Review Text"]=df['Review Text'].str.lower() df["Review Text"]=df['Review Text'].str.replace(r'[-./?!,":;()\']',' ')
The first line in the code above lower cases all words. The second line removes the punctuation. The second line is trickier as you have to explain to python exactly what type of punctuation you want to remove and what to replace it with. Everything we want to remove is in the first set of single quotes. We want to replace the punctuation with a space which is the second set of single quotation marks with a space in the middle. THe r at the beginning of the parentheses stands for remove.
Here is what our data looks like after making these two changes
df['Review Text'].head() Out[249]: 0 absolutely wonderful silky and sexy and comf... 1 love this dress it s sooo pretty i happene... 2 i had such high hopes for this dress and reall... 3 i love love love this jumpsuit it s fun fl... 4 this shirt is very flattering to all due to th... Name: Review Text, dtype: object
All the words are in lowercase. In addition, you can see that the dash in line 0 is gone as all the punctuation in the other lines. We now need to remove stopwords. Stopwords are the functional words that glue the meaning together without. Examples include and, for, but, etc. We are trying to make a cloud of substantial words and not stopwords so these words need to be removed.
If you have never done this on your computer before you may need to import the nltk module and run nltk.download_gui(). Once this is done you need to download the stopwords package.
Below is the code for removing the stopwords. First, we need to load the stopwords this is done below.
stopwords_list=stopwords.words('english')
stopwords_list=stopwords_list+['to']
We create an object called stopwords_list which has all the English stopwords. The second line just adds the word ‘to’ to the list. Nex,t we need to make an object that will look for the pattern of words we want to remove. Below is the code
pat = r'\b(?:{})\b'.format('|'.join(stopwords_list))
This code is the basically telling Python what to look for. Using regularized expressions Python will look for any word whos pattern on the left is the same as the pattern on the right after the .join function. Inside the .join function is our stopwords_list. We will now take this object called ‘pat’ and use it on our ‘Review Text’ variable.
df['Split Text'] = df['Review Text'].str.replace(pat, '') df['Split Text'].head() Out[258]: 0 absolutely wonderful silky sexy comfortable 1 love dress sooo pretty happened find ... 2 high hopes dress really wanted work ... 3 love love love jumpsuit fun flirty f... 4 shirt flattering due adjustable front t... Name: Split Text, dtype: object
You can see that we have created a new column called ‘Split Text’ and the results is a text that has lost many stop words.
We are now ready to make our word cloud below is the code and the output.
wordcloud1=wordcloud.WordCloud(width=1000,height=500).generate(' '.join(map(str, df['Split Text'])))
plt.figure(figsize=(15,8))
plt.imshow(wordcloud1)
plt.axis('off')
This code is complex. We used the word cloud function and we had to use both generate map, and join as inner functions. All of these function were needed to take the words from the dataframe and make them simple text for the wordcloud function.
The rest of the code is common to mathplotlib so does not require much explanation. Ass you look at the word cloud, you can see that the most common words include top, look, dress, shirt, fabric. etc. This is reasonable given that these are women’s reviews of clothing.
Conclusion
This post provided an example of text analysis using word clouds in Python. The insights here are primarily descriptive in nature. This means that if the desire is prediction or classification other additional tools would need to build upon what is shown here.