Appendix A — Practice: Hypothesis Testing

The objective of this practice is to consolidate the knowledge acquired in parametric and non-parametric hypothesis testing, apply various statistical tests to different types of data, and interpret the results of these tests to make data-driven decisions.

A.1 Deliverables

You must submit a PDF document containing the entire data analysis process related to the questions posed in the Exercises section below.

To generate the PDF document, you must use R Markdown or Quarto. This allows you to easily interweave explanations and code with results.

The PDF document will be uploaded to a task in the virtual campus.

A.2 Suggested Workflow

For each dataset, it is recommended to follow these analysis steps:

  1. Data exploration: Use functions like summary, str, head, tail, and graphs to familiarize yourself with the data.
  2. Data cleaning: Identify and handle missing values, outliers, and any other data issues. Note: this step is not necessary for this practice.
  3. Hypothesis formulation: Clearly define the null and alternative hypotheses you want to test.
  4. Statistical test selection: Choose the appropriate test based on the data type and hypotheses.
  5. Test execution: Use R functions to perform the statistical test.
  6. Result interpretation: Analyze the confidence interval and other parameters to make a decision about the null hypothesis.
  7. Visualization: Use graphs to effectively communicate the results.

A.3 Datasets

This section describes the datasets used in this practice, as well as their online availability.

A.3.1 Titanic

A classic dataset that offers a rich variety of categorical and numerical variables.

The Titanic dataset is a collection of information about the passengers and crew aboard the famous transatlantic liner that sank in the North Atlantic in 1912. This dataset has become a classic in the field of data science and machine learning, frequently used to teach data analysis techniques and to explore research questions related to survival, socioeconomic factors, and decision-making in critical situations.

What does the dataset contain?

The Titanic dataset typically includes variables such as:

  • Passenger information: Name, age, sex, social class (first, second, third), whether they had a partner or family members on board, ticket fare, port of embarkation, among others.
  • Survival information: Whether the passenger survived the shipwreck or not.

What is the dataset used for?

This dataset is used for various purposes, including:

  • Exploratory data analysis: To understand the characteristics of the passengers and their relationship to survival.
  • Predictive modeling: To build models that predict the probability of a passenger surviving based on their characteristics.
  • Data visualization: To create graphs and visualizations that illustrate the relationships between variables and tell the story of the disaster.

Download:

Available from [https://www.kaggle.com/c/titanic/data].

A.3.2 HR Analytics Job Prediction

This dataset is an invaluable tool for analyzing various factors that influence job satisfaction and employee turnover in an organization. Like the Titanic dataset, it has become a benchmark for those who want to learn and practice data analysis techniques in the field of human resources.

What does the dataset contain?

The HR Analytics Job Prediction dataset typically includes variables such as:

  • Employee information: Age, gender, education level, department, role, years of experience, salary, etc.
  • Performance information: Performance evaluation, number of projects, hours worked, etc.
  • Satisfaction information: Level of job satisfaction, satisfaction and dissatisfaction factors, etc.
  • Turnover information: Whether the employee left the company or not.

What is the dataset used for?

This dataset is used for various purposes, including:

  • Exploratory data analysis: To understand the characteristics of employees and their relationship to job satisfaction and turnover.
  • Predictive modeling: To build models that predict the probability of an employee leaving the company based on their characteristics.
  • Human resources optimization: To identify the factors that most influence job satisfaction and take measures to improve the work environment.

Download:

Available at this Kaggle page.

A.4 Functions to use

The following functions will be the basics for performing the proposed analyses.

  • t.test: To compare means of one or two samples.
  • wilcox.test: To compare means of one or two samples when normality is not assumed.
  • var.test: To compare variances of two samples.
  • prop.test: To compare proportions of one or two samples.
  • binom.test: To test hypotheses about a proportion of a binomial sample.
  • chisq.test: To test the independence between two categorical variables or goodness of fit.
  • ks.test: To test if two samples come from the same distribution.

It is recommended that you use R’s help to learn how to use them, and seek help online (or from the professor) on when and how to use each function.

A.5 Exercises

Here are some questions that can be answered using the hypothesis testing tools seen in class.

The idea is that each question translates into an analysis (hypothesis test) and this translates into a series of R commands, using the functions already mentioned and other auxiliary ones.

A.5.1 Titanic

Analyse the following questions.

Survival

  1. Is the proportion of survivors significantly different from 50%? Hint: Use binom.test.
  2. Is there a significant difference in the average age of survivors and the deceased? Hint: Use t.test or wilcox.test as appropriate.
  3. Are the variances of age equal between survivors and deceased? Hint: Use var.test.

Social class

  1. Is the distribution of social class among passengers uniform? Hint: Use chisq.test.
  2. Is there a significant difference in the proportion of passengers traveling alone among the different social classes? Hint: Use prop.test.

Other questions of interest

  1. Comparison of ages: Is there a significant difference in the average age of passengers traveling in first and third class?
  2. Survival by gender and class: Does the interaction between gender and social class influence the probability of survival?
  3. Comparison of fares: Is there a significant difference in the average fares paid by passengers who survived and those who died?

A.5.2 HR Analytics Job Prediction

This dataset allows us to analyze various factors that influence job satisfaction and employee turnover.

Satisfaction and performance evaluation

  1. Is there a significant difference in the level of satisfaction between employees with a high performance evaluation and those with a low evaluation?
  2. Do employees with an outstanding performance evaluation have a significantly higher probability of being satisfied with their job?

Hours worked and satisfaction

  1. Are employees who work more hours significantly less satisfied with their job?
  2. Is there a significant difference in the level of satisfaction between employees who work full-time and those who work part-time?

Number of projects and satisfaction

  1. Is there a negative relationship between the number of projects assigned to an employee and their level of satisfaction?
  2. Are employees who work on more than three projects simultaneously significantly less satisfied?

Employee turnover and satisfaction

  1. Did employees who decided to leave the company have a significantly lower level of satisfaction before leaving?
  2. Is there a significant difference in the level of satisfaction between employees who stayed with the company and those who left?

A.6 Additional tasks

As a complement to the previous discussions, it is advisable to consider the following:

  1. Visualizations: Use the {ggplot2}📦 to create histograms, boxplots, and other graphs to help visualize the data and the results of the contrasts.

  2. Confidence intervals: Construct confidence intervals for the estimated parameters to support the explanations given.