LIS 4370 R Programming - Module 10 Assignment

 sentimentTextAnalyzer R Package Proposal

Introduction:

The sentimentTextAnalyzer package aims to provide a comprehensive tool for analyzing textual data to extract sentiment insights. With the increasing volume of text data generated on various platforms, understanding sentiment is crucial for businesses and researchers alike. sentimentTextAnalyzer offers a robust solution for sentiment analysis, enabling users to extract positive and negative sentiment signals from diverse sources such as URLs and flat text files.

Objectives:

• Develop a versatile R package, sentimentTextAnalyzer, capable of analyzing text data for positive and negative sentiment.
• Implement algorithms to parse text efficiently and extract sentiment insights.
• Enable the package to generate word clouds highlighting the most frequent words in the text data.

Key Features:

• Text Parsing: Implement algorithms to parse text from various sources, including URLs and flat text files.
• Sentiment Analysis: Develop algorithms to identify positive and negative sentiment words and calculate their frequency in the text.
• Word Cloud Customization: Enable the package to generate word clouds depicting the most common words in the text data.
• Customization: Allow users to customize sentiment analysis parameters and word cloud generation options.

Methodology:

• Text Parsing: Utilize natural language processing (NLP) techniques to preprocess and tokenize text data. 
• Sentiment Analysis: Implement sentiment lexicons and algorithms to identify positive and negative sentiment words.
• Word Cloud Generation: Utilize packages such as wordcloud2 to generate visually appealing word clouds based on word frequency. 
• Package Development: Utilize R programming language and relevant packages (e.g., tidyverse, text mining) to develop the sentimentTextAnalyzer package. 
• Testing and Validation: Conduct thorough testing and validation to ensure the accuracy and reliability of sentiment analysis results.

Following this link will take you to the package's description file which provides a few details regarding licensing, potential dependencies, and the current version of the package: DESCRIPTION

~ Katie

LIS 4370 R Programming - Module 9 Assignment

In this week in R Programming, we are asked to select a dataset from the Vincent Arel Bundock dataset list and create visualizations from that data.

Link to the list: Vincent Arel Bundock Datasets

I decided to work from the pizzaplace.csv dataset which contains sales, pizza type, and size data over an entire year. 

In the instructions, it is mentioned that there are three ways to make visualizations in R: Base R, Lattice package, and ggplot2 package. Thus, I will generate visualizations from these listed methods.

In base R, let's make a "pie" chart that determines the occurrences of each of the 4 types of pizzas sold:

As for the second visual, let's use the lattice package to explore the relationship between prices and pizza size:

Moving on to the third visual using the ggplot2 package, the data was separated by facet to make it easier to compare trends between the four types of pizza and their associated sales over time:

After creating the visuals using the three different methods, I must admit that it is interesting to see how each method does have its pros and cons. For example, I do like to use the base R method but things can get complicated fast with having to call out all the individual methods. To make the first visual better, I should include percentages for each of the four types of pizza sold. Moving on to the second visual, I do not have too much experience using the Lattice package but I do think that the visual came out well in telling a story with the data. For instance, it still weirds me out that someone bought a super expensive small pizza that surpassed the price of a large pizza. Lastly, the ggplot2 visual really puts into perspective which pizza type is the most expensive in terms of sales like classic going above 30. 

Check out the code here: Module 9 Code

~ Katie 

LIS 4317 Visual Analytics - Module 9 Assignment

In this week of Visual Analytics, we are asked to create a multi-variate visualization graph with a dataset of our choice.

In this case, I decided to work with a dataset called nyc_squirrels.csv and basically it contains very detailed observations from squirrel watching in New York City's Central Park. From what they were doing, what sound they made, to even the exact geo coordinates of where the squirrel watching event occurred, it is all noted down in the data. 

Link to where I found the data: NYC Squirrels Data

From this data, I decided that I wanted to better understand the spatial distribution of squirrel sightings and see if there is any difference in sightings that occurred in the AM or PM.

To begin my analysis, I did make a point to clean my data containing entries with NA values and deleted variables that were not conducive to the analysis.

With the data ready, I used the ggplot2 package in R to graph the points:

Here is the visual: 

As you can see, when the geo coordinate points are plotted, it actually makes a rough outline of Central Park. The big empty gap you are seeing represents Jacqueline Kennedy Onassis Reservoir so it makes sense that there were not any squirrels spotted there. For the most part, I do not see any particular difference in squirrel sightings in the AM versus PM but there does appear to be more squirrel sightings at night than during the day.

For fun, let's see what this plot looks like in Tableau with a map underneath the points:

See the map up close here: NYC Squirrel Sightings

Wrapping up, visualizing multi-variables can be very helpful when it comes to understanding the subtle relationships between them. It is definitely interesting to be able to compare AM sightings to PM sightings and where they occurred in Central Park and allows for one to better understand the dataset.

As for applying the 5 principles of design, alignment is used for the axis labels, legend, and title for better readability. With repetition, shape style, color, font size, and type are kept consistently. To highlight the difference between day and night, I opted to use cool colors most often associated with the night for PM and warm colors for AM which checks off the contrast requirement. Moving on to proximity, visual elements like the legend are clearly placed together to promote connection. Lastly, with balance, I must admit that the Tableau visual is not as balanced as the previous ggplot visual. It has very small legend which makes it have uneven weight. To prevent this, I should think about adding more data elements to make the visual more balanced.

~ Katie

LIS 4317 Visual Analytics - Module 8 Assignment

In this week in Visual Analytics, we are asked to generate a visual from the mtcars dataset that is either based on correlation and regression analysis.

In this case, I will be conducting correlation analysis on the variables number of cylinders (cyl) and horsepower (hp). That is, I want to know if these variables have a positive correlation or negative correlation relationship.

Let's begin by plotting the relationship via ggplot2 in a scatterplot. I will also draw in a line of best fit using the lm argument.

Here's my code:

The resulting output:

Interestingly, there appears to be a rather strong positive correlation between the number of cylinders and horsepower. Perhaps, this is alluding to cars that have a greater number of cylinders have a higher horsepower. Let's take a look at the correlation coefficient.

With a value of .832, that's a pretty high correlation value!

Lastly, let's finish up by creating a linear regression model of these variables:

Looking at the significance codes, cyl is highly significant at the (***) mark at 0.001. However, the model can only capture about 68% of the variability so it is a moderately strong correlation.

Reflecting on the design remarks of Few from the textbook, my visual does follow a few of his best practices. To begin, I do feature some grid lines for greater readability of graph and one can easily determine the points and where they fall in terms of cyl and hp. However, I do see how grid lines can be helpful if multiple scatter plots are used in the analysis. In addition, my visual also has a line of best fit so that we easily see the correlation's linear shape and positive slope.

I appreciate Few's recommendations when it comes to visualizing correlations between variables. Although my visual does not include all his suggestions and I know that they will be helpful for other datasets.

~ Katie

LIS4370 R Programming - Module 8 Assignment

For this week's assignment in R Programming, we are to do the following:

Step 1: Import txt file in R. This file is called Assignment 6 Dataset.txt

Step 2: Use ddply from the plyr package to generate mean of Age and Grade variables split by Sex variable, this will be saved to new variable, Grade.Average

Step 3: Create a new txt file containing new variable Grade.Average. This new txt file will be called Sorted.Average

Step 4: Separate values in Sorted.Average by comma using sep argument, and save it back to Sorted.Average

Step 5: Using the original txt file, filter the names in the list that contain the letter (i). Then, save the result to a new file called DataSubset with the values separated by comma.

Let's take a look at our new files:

Sorted.Average:

DataSubset:

As you can see by the resulting output, the values are separated by commas and the variables are differentiated as they are surrounded by double quotes. 

See the code on GitHub: Module 8 Code

~ Katie

LIS 4317 Visual Analytics - Module 7 Assignment

For this week's assignment, we are tasked with creating visual analytics based on distribution analysis. I will be working with the mtcars dataset to understand the distribution of horsepower (hp).

A quick note, I generated a couple visuals of the horsepower (hp) distribution:

Scatter plot:

Boxplot:

Line Graph:

Histogram:

Reflecting on Few's recommendations in testing and best practices when it comes to conducting distribution analysis, each of my graphs have strengths and weaknesses. To begin, Few notes that there are three main characteristics when it comes to describing distributions. These are...

Spread: A simple measure of dispersion, or how spread out the values are and it essentially is the full range of values from highest to lowest.

Center: An estimate of the middle of a set of values and it is often demonstrated by either the mean or median.

Shape: Where the values are located throughout the spread. 

For the most part, my visuals do a good job of showing spread except for maybe the boxplot as it simplifies the values that are shown on the y-axis tick marks but the full spread is still albeit it is slightly downplayed. As for center, visuals 1, 3, and 4 provides horsepower's mean and median and where it lies in correspondence to the chart. The second visual, the boxplot, only provides the median. Moving on to shape, one can note that visuals 1, 3, and 4 appear slightly skewed to the right. In the histogram, one can also see a brief gap near the 300 tick mark and an outlier when hp equals 325. 

As for whether these visuals correspond to Few's distribution analysis best practices, I believe my visuals do a fairly good job when it comes to interval consistency but fails when it comes to outlier resistance. As one can tell from the visuals, there is a clear outlier where hp equals 325. The mean calculation can be heavily affected by outliers and as a result, can be shifted in the direction of that outlier and we can clearly see that happening here. Therefore, it might be a good idea to remove that outlier from the dataset before conducting visual analysis. 

All in all, Few's recommendations are incredibly helpful when it comes to deciphering data when it is visualized.

~ Katie

LIS 4370 R Programming - Module 7 Assignment

For this week's assignment, I will start out by examining the iris dataset and then transition to my own dataset when it comes to creating two examples of S3 and S4.

Question 1: Determine if a generic function can be applied to your dataset

To begin, I used the following functions on the iris dataset and came up with the following output:

Based on this output, I can confirm that a generic function can be applied to my chosen dataset.

Question 2: How do you tell what OO system (S3 vs. S4) an object is associated with?

In the library, pryr, one can use the function otype() to determine which OO system object is associated with.

In this case, the iris dataset is associated with S3.

Question 3: How do you determine what the base type of an object is?

Using the typeof() function can help in determining an object's base type. Continuing with the iris dataset, we can check the object type of each of the variables within the dataset:

Question 4: What is an generic function?

A generic function can be defined as a function that performs a common task like printing (print()) or even plotting (plot()). Furthermore, they can be thought as extended function objects because they contain information that is used in creating and dispatching for the function.

Question 5: What are the main differences between S3 and S4?

To put it simply, S3 is considered more convenient while S4 is more safe. Additionally, S3 classes are very straightforward to implement as it only uses the first argument to dispatch but it can allow for mistakes to slip through like misspelled values and missing values and will not alert the programmer of the potential issues. On the other hand, S4 classes and methods are way more formal and more closely related to object-oriented concepts and unlike S3, S4 will complain about such misspellings and other issues to alert the programmer that the current code does need to be fixed.

Question 6: Create two examples of S3 and S4. (Code will be linked to GitHub)

S3 Code:

Output:

S4: Code:

Output:

After conducting this brief code experiment, I must admit that I greatly prefer the form of S4 over S3 just for its ease of creating instances of the class. 

Link to GitHub Code: Module 7 Code

~ Katie