Cleaning data

Data set used - mental health in IT, available on Kaggle

Data can be messy. The data set above for example had a 'Gender' field which contained many variations on male and female, for example: Male, male, M, m, man, F, f, Female and so on. The first thing I wanted to do was set all values to either male or female. One way to do this is with the following code:

​import pandas as pd

df = pd.read_csv('survey.csv')

df = df.replace({'Gender' : { 'Male' : 'male', 'M' : 'male', 'm' : 'male' }})
df = df.replace({'Gender' : { 'Female' : 'female', 'F' : 'female', 'f' : 'female' }})