Intro to Scikit learn¶
Machine learning: the problem setting¶
| outlook | temperature | humidity | wind | play |
|---|---|---|---|---|
| sunny | hot | high | false | no |
| sunny | hot | high | true | no |
| overcast | hot | high | false | yes |
| rainy | mild | high | false | yes |
| rainy | cold | normal | false | yes |
| rainy | cold | normal | true | no |
| overcast | cold | normal | true | yes |
| sunny | mild | high | false | ? |
- learning problem considers a set of n samples of data which have m features (e.g. outlook,temperature, etc.)
- tries to predict properties of unknown data
- We can separate it in two categories:
- supervised learning: predict additional attributes
- classification: discrete values (e.g. play)
- regression: continuous values (e.g. temperature in degree celsius)
- unsupervised learning: e.g. find clusters of "similar" days
- supervised learning: predict additional attributes
Structure Data¶
- the data is an array (n_samples, n_features)
[[ 0. 0. 5. ..., 0. 0. 0.] [ 0. 0. 0. ..., 10. 0. 0.] [ 0. 0. 0. ..., 16. 9. 0.] ..., [ 0. 0. 1. ..., 6. 0. 0.] [ 0. 0. 2. ..., 12. 0. 0.] [ 0. 0. 10. ..., 12. 1. 0.]] - in case of supervised learning, we also have a target variable (n_samples)
[ True, False, True, True, False ...]
In [1]:
import pandas as pd
customer_data = pd.read_excel('CustomerDataSet.xls')
customer_data.head()
Out[1]:
In [2]:
from sklearn.cluster import KMeans
estimator = KMeans(n_clusters = 2)
labels = estimator.fit_predict(customer_data[['ItemsBought', 'ItemsReturned']])
print(labels)
# OR
estimator.fit(customer_data[['ItemsBought', 'ItemsReturned']])
print(estimator.labels_)
Plotting clusters:¶
For plotting the clusters you can use the c parameter of the plotting function
In [3]:
import matplotlib.pyplot as plt
plt.title("KMeans #cluster = 2")
plt.xlabel('ItemsBought')
plt.ylabel('ItemsReturned')
plt.scatter(customer_data['ItemsBought'], customer_data['ItemsReturned'], c=estimator.labels_)
plt.show()
Calculating the silhouette score¶
In [4]:
from sklearn.metrics import silhouette_score
silhouette = silhouette_score( customer_data[['ItemsBought', 'ItemsReturned']], labels)
silhouette
Out[4]:
Preprocess / Encoding¶
- most maschine learning algorithmns can only work with numerical data
- to "translate" text data to numerical data, one can use the LabelEncoder
In [5]:
print(customer_data[['Product', 'ZipCode']])
from sklearn import preprocessing
customer_data_encoded = customer_data[['Product', 'ZipCode']].apply(preprocessing.LabelEncoder().fit_transform)
print(customer_data_encoded[['Product', 'ZipCode']])
Preprocess/ Normalise¶
- possibilities for normalisation
- StandardScaler (creates zero mean and unit variance)
- MinMaxScaler ( e.g. scale to [0,1] )
In [6]:
from sklearn import preprocessing
#inplace
#customer_data[['ItemsBought', 'ItemsReturned']] = preprocessing.MinMaxScaler().fit_transform(customer_data[['ItemsBought', 'ItemsReturned']])
#creating a new dataframe
customer_data_normalised = pd.DataFrame(
preprocessing.MinMaxScaler().fit_transform(customer_data[['ItemsBought', 'ItemsReturned']]),
columns=['ItemsBought', 'ItemsReturned']
)
In [7]:
print(customer_data[['ItemsBought', 'ItemsReturned']])
print(customer_data_normalised)
Pandas¶
- Extending the data your your own values is done with
In [8]:
customer_data_with_cluster = customer_data.assign(cluster=estimator.labels_)
customer_data_with_cluster
Out[8]:
Agglomerative Hierarchical Clustering (for nice diagrams):¶
- using the linkage function from scipy
- linkage(data, method, metric)
- method parameter can be one of "single", "complete", "average", "weighted", "centroid", "median", "ward"
- metric parameter (default is okay for us) can be one of "euclidean", "minkowski", "cityblock", "seuclidean", "hamming", "jaccard" and others (see pdist function
- plotting is done with the dendrogram function
- dendrogram(Z) Z is the result of the linkage function
- Having nice labels is done with
dendrogram(Z,labels=data['column'].values)
In [9]:
from scipy.cluster.hierarchy import dendrogram, linkage
Z = linkage(customer_data[['ItemsBought', 'ItemsReturned']], 'ward')
plt.title('Hierarchical Clustering Dendrogram')
plt.xlabel('Customer IDs')
plt.ylabel('distance')
dendrogram(Z, labels=customer_data['Customer ID'].values)
plt.show()
Truncate the dendrogram¶
- setting truncate_mode to 'lastp'
- set the p attribute to the count of clusters
In [10]:
plt.title('Dendrogram - 3 clusters')
plt.xlabel('Count of Customers')
plt.ylabel('distance')
dendrogram(Z,
truncate_mode='lastp',
p=3)
plt.show()
Changing the orientation of the dendrogram is done with orientation
In [11]:
ax = dendrogram(
Z,
orientation='right'
)
plt.show()
Agglomerative Hierarchical Clustering (for cluster values):¶
- like KMeans
In [12]:
from sklearn.cluster import AgglomerativeClustering
agg = AgglomerativeClustering(n_clusters = 3)
agg_predictions = agg.fit_predict(customer_data[['ItemsBought', 'ItemsReturned']])
plt.scatter(customer_data['ItemsBought'], customer_data['ItemsReturned'], c=agg_predictions)
plt.show()
DBScan:¶
- nearly the same as KMeans
In [13]:
from sklearn.cluster import DBSCAN
db = DBSCAN().fit(customer_data[['ItemsBought', 'ItemsReturned']])
plt.scatter(customer_data['ItemsBought'], customer_data['ItemsReturned'], c=db.labels_)
plt.show()
Load Data (arff)¶
- just use the arff function from scipy
In [14]:
from scipy.io import arff
zoo_arff_data, zoo_arff_meta = arff.loadarff(open('zoo.arff', 'r'))
zoo_data = pd.DataFrame(zoo_arff_data)
zoo_data.head()
Out[14]:
Creating multiple plots¶
In [16]:
#for i in range(2,5):
for i in [2,3,4]:
estimator = KMeans(n_clusters = i)
estimator.fit(customer_data[['ItemsBought', 'ItemsReturned']])
plt.title("#cluster = {}".format(i))
plt.scatter(customer_data['ItemsBought'], customer_data['ItemsReturned'], c=estimator.labels_)
plt.show()
Subplots (side by side)¶
In [34]:
plt.figure(1, (10,10)) # otherwise the image is really small
counter = 1 # we need a counter to start at one
for i in [2,3,4,5,6]:
plt.subplot(3,2,counter) # plt.subplot(rows, columns, current_index)
#plt.tight_layout() # sometimes you need it, when the plots are a bit overlapping
counter += 1
estimator = KMeans(n_clusters = i)
estimator.fit(customer_data[['ItemsBought', 'ItemsReturned']])
plt.title("#cluster = {}".format(i))
plt.scatter(customer_data['ItemsBought'], customer_data['ItemsReturned'], c=estimator.labels_)
plt.show()
