机器学习——决策树的实现

#!/usr/bin/env python
#-*-coding:utf-8-*-
#决策树的建立，训练测试，
from sklearn.feature_extraction import DictVectorizer
import csv
from sklearn import preprocessing
from sklearn import tree
from sklearn.externals.six import StringIO#读数据
allElectronicsData=open(r'jueceshu.csv','rb')
reader=csv.reader(allElectronicsData)
headers=reader.next()featureList=[]
labelList=[]
#分析数据
for row in reader:#print(row)if(row):labelList.append(row[len(row)-1])rowDict={}for i in range(1,len(row)-1):rowDict[headers[i]]=row[i]featureList.append(rowDict)print(featureList)
#转化数据
vec=DictVectorizer()
dummyX=vec.fit_transform(featureList).toarray()print('dummyX:'+str(dummyX))
print(vec.get_feature_names())print('labelList:'+str(labelList))lb=preprocessing.LabelBinarizer()
dummyY=lb.fit_transform(labelList)
print('dummyY:'+str(dummyX))#训练数据
clf=tree.DecisionTreeClassifier(criterion='entropy')
clf=clf.fit(dummyX,dummyY)
print('clf'+str(clf))
#转化为dot模式
with open('allElectronicInformationGainDri.dot','w') as f:f=tree.export_graphviz(clf,feature_names=vec.get_feature_names(),out_file=f)#决策树的预测
oneRowX=dummyX[0,:]
print('oneRowX:'+str(oneRowX))newRowX=oneRowX
newRowX[0]=1
newRowX[2]=0
print('newRowX:'+str(newRowX))predictedY=clf.predict(newRowX)
print('predictedY:'+str(predictedY))

student=no <= 0.5

entropy = 0.9403
samples = 14
value = [5, 9]
True
entropy = 0.0
samples = 5
value = [0, 5]
False
age=senior <= 0.5
entropy = 0.5436
samples = 8
value = [1, 7] age=youth <= 0.5
entropy = 0.9183
samples = 6
value = [4, 2]
credit_rating=excellent <= 0.5
entropy = 0.9183
samples = 3
value = [1, 2] credit_rating=excellent <= 0.5
entropy = 0.9183
samples = 3
value = [1, 2]
entropy = 0.0
samples = 2
value = [0, 2]
entropy = 0.0
samples = 1
value = [1, 0]
entropy = 0.0
samples = 2
value = [0, 2]
entropy = 0.0
samples = 3
value = [3, 0]
entropy = 0.0
samples = 1

value = [1, 0]

1. Python

2. Python机器学习的库：scikit-learn

2.1：特性：

简单高效的数据挖掘和机器学习分析

对所有用户开放，根据不同需求高度可重用性

基于Numpy, SciPy和matplotlib

开源，商用级别：获得 BSD许可

2.2 覆盖问题领域：

分类（classification), 回归（regression), 聚类（clustering), 降维(dimensionality reduction)

模型选择(model selection), 预处理(preprocessing)

3. 使用用scikit-learn

安装scikit-learn: pip, easy_install, windows installer

安装必要package：numpy， SciPy和matplotlib，可使用Anaconda (包含numpy, scipy等科学计算常用

package）

安装注意问题：Python解释器版本（2.7 or 3.4？）, 32-bit or 64-bit系统

4. 例子：

文档： http://scikit-learn.org/stable/modules/tree.html

解释Python代码

安装 Graphviz： http://www.graphviz.org/

配置环境变量

转化dot文件至pdf可视化决策树：dot -Tpdf iris.dot -o outpu.pdf

数据集：

RiD,age,income,student,credit_rating,Class_buys_computer
1,youth,high,no,fair,no
2,youth,high,no,excellent,no
3,middle_aged,high,no,fair,yes
4,senior,medium,no,fair,yes
5,senior,low,yes,fair,yes
6,senior,low,yes,excellent,no
7,middle_aged,low,yes,excellent,yes
8,youth,medium,no,fair,no
9,youth,low,yes,fair,yes
10,senior,medium,yes,fair,yes
11,youth,medium,yes,excellent,yes
12,middle_aged,medium,yes,excellent,yes
13,middle_aged,high,yes,fair,yes
14,senior,medium,no,excellent,no