杭州笕桥网站建设,做评选活动的网站,网站与域名的关系,小程序制作流程微信目录
一、简介和环境准备
二、算法简介 2.1四种方法类#xff1a;
2.1.1FKNN
2.1.2FKNCN
2.1.3BM-FKNN 2.1.3BM-FKNCN
2.2数据预处理
2.3输出视图
2.4调用各种方法看准确率
2.4.1BM-FKNCN
2.4.2BM-FKNN
2.4.3FKNCN
2.4.4FKNN
2.4.5KNN 一、简介和环境准备
k…目录
一、简介和环境准备
二、算法简介 2.1四种方法类
2.1.1FKNN
2.1.2FKNCN
2.1.3BM-FKNN 2.1.3BM-FKNCN
2.2数据预处理
2.3输出视图
2.4调用各种方法看准确率
2.4.1BM-FKNCN
2.4.2BM-FKNN
2.4.3FKNCN
2.4.4FKNN
2.4.5KNN 一、简介和环境准备
knn一般指邻近算法。 K最近邻算法是一种常见的监督式学习算法用于分类和回归问题。在K最近邻算法中给定一个新的数据点算法会找到训练数据集中离这个数据点最近的K个数据点然后使用这K个数据点的标签或属性来预测新数据点的标签或属性。
主角是一种基于局部Bonferroni均值的模糊K-最近质心近邻BM-FKNCN分类器。下文会详细介绍BM-FKNCN。
本次实验环境需要用的是Google Colab和Google Drive云盘文件后缀是.ipynb可以直接用。首先登录谷歌云盘网页再打卡ipynb文件就可以跳转到谷歌colab了。再按以下点击顺序将colab和云盘链接。 输入依赖
from google.colab import drive
import pandas as pd
import numpy as np
import scipy.spatial
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import StratifiedKFold
from sklearn.model_selection import KFold
from sklearn.metrics import confusion_matrix
from sklearn.metrics import plot_confusion_matrix
import matplotlib.pyplot as plt
drive.mount(/content/drive) 二、算法简介 2.1四种方法类
FKNN、FKNCN、BM_FKNN和BM_FKNCN之后再调用不用先运行
2.1.1FKNN
FKNN是指Fuzzy K-Nearest Neighbor模糊K最近邻算法。它使用模糊逻辑来考虑数据点之间的相似性。在FKNN中每个数据点都被赋予一个隶属度membership degree该隶属度表示数据点属于每个类别的可能性程度。与传统的K最近邻算法不同FKNN不仅考虑最近的K个数据点还考虑了与目标数据点在一定距离范围内的所有数据点。FKNN的主要优点是能够处理数据集中的噪声和模糊性并且对于不平衡的数据集也表现良好。
#Methods##FKNNimport scipy.spatial
from collections import Counter
from operator import itemgetterclass FKNN:def __init__(self, k):self.k kdef fit(self, X, y):self.X_train Xself.y_train ydef getDistance(self, X1, X2):return scipy.spatial.distance.euclidean(X1, X2)def fuzzy(self, d, m):closestPoint [ d[k][1] for k in range(len(d))]classes np.unique(self.y_train[closestPoint])arrMembership []for cls in classes:atas 0bawah 0for close in d: if(close[0] ! 0):if(cls self.y_train[close[1]]):atas np.power((1/close[0]), (2/(m-1)))else:atas np.power((0/close[0]), (2/(m-1)))bawah np.power((1/close[0]), (2/(m-1)))else:atas 0bawah 1arrMembership.append([atas/bawah, cls])return arrMembershipdef predict(self, X_test):final_output []for i in range(len(X_test)):d []votes []for j in range(len(X_train)):dist self.getDistance(X_train[j] , X_test[i])d.append([dist, j])d.sort()d d[0:self.k]membership self.fuzzy(d, 2)predicted_class sorted(membership, keyitemgetter(0), reverseTrue)final_output.append(predicted_class[0][1]) return final_outputdef score(self, X_test, y_test):predictions self.predict(X_test)value 0for i in range(len(y_test)):if(predictions[i] y_test[i]):value 1return value / len(y_test)
2.1.2FKNCN
模糊K-最近质心近邻FKNCN分类器是一种基于邻近性的模糊分类算法是模糊K-最近邻FKNN算法的一种变体。与FKNN类似FKNCN算法也使用模糊理论中的隶属度来度量样本之间的相似度。但是与FKNN不同的是FKNCN算法不仅考虑最近邻居的距离还考虑了它们的质心。具体来说对于每个测试样本FKNCN算法首先使用K-最近邻算法来找到其K个最近的邻居。然后对于每个类别FKNCN算法计算其K个最近邻居的质心并将测试样本与每个质心之间的距离作为该类别的隶属度。最后FKNCN算法通过计算每个类别的隶属度之和来确定测试样本所属的类别。
与FKNN相比FKNCN具有以下优点 对异常值和噪声更加鲁棒FKNCN算法不仅考虑到每个邻居的距离还考虑到它们的质心因此对异常值和噪声更加鲁棒。 对于类别不平衡的数据集有更好的性能FKNCN算法可以根据每个类别的质心来调整类别之间的权重因此可以处理类别不平衡的数据集。 算法相对简单FKNCN算法比一些更复杂的算法如支持向量机具有更简单的实现和计算。
##FKNCNimport scipy.spatial
from collections import Counter
from operator import itemgetterclass FKNCN:def __init__(self, k):self.k kdef fit(self, X, y):self.X_train Xself.y_train ydef getDistance(self, X1, X2):return scipy.spatial.distance.euclidean(X1, X2)def getFirstDistance(self, X_train, X_test):distance []for i in range(len(X_train)):dist scipy.spatial.distance.euclidean(X_train[i] , X_test)distance.append([i, dist, self.y_train[i]])return distancedef getCentroid(self, arrData):result[]dataTran np.array(arrData).Tfor i in range(len(dataTran)):result.append(np.mean(dataTran[i]))return resultdef kncn(self, X_test):myclass list(set(self.y_train))closestPoint []anothersPoint []for indexK in range(0, self.k):if(indexK 0):distance self.getFirstDistance(self.X_train, X_test) distance_sorted sorted(distance, keyitemgetter(1))closestPoint.append(distance_sorted[0])distance_sorted.pop(0)for anothers in (distance_sorted):anothersPoint.append(anothers[0]) else:arrDistance []closestPointTemp [self.X_train[r[0]] for r in closestPoint]for r in (anothersPoint):arrQ closestPointTemp.copy()arrQ.append(self.X_train[r])arrDistance.append([r, self.getDistance(self.getCentroid(arrQ), X_test)])distance_sorted sorted(arrDistance, keyitemgetter(1))closestPoint.append(distance_sorted[0])# anothersPoint np.setdiff1d(anothersPoint, closestPoint)return closestPointdef fuzzy(self, d, m):closestPoint [ d[k][1] for k in range(len(d))]classes np.unique(self.y_train[closestPoint])arrMembership []for cls in classes:atas 0bawah 0for close in d: if(close[0] ! 0):if(cls self.y_train[close[1]]):atas np.power((1/close[0]), (2/(m-1)))else:atas np.power((0/close[0]), (2/(m-1)))bawah np.power((1/close[0]), (2/(m-1)))else:atas 0bawah 1arrMembership.append([atas/bawah, cls])return arrMembershipdef predict(self, X_test):final_output []for i in range(len(X_test)):closestPoint self.kncn(X_test[i])d []votes []for j in range(len(X_train)):dist self.getDistance(X_train[j] , X_test[i])d.append([dist, j])d.sort()d d[0:self.k]membership self.fuzzy(d, 2)predicted_class sorted(membership, keyitemgetter(0), reverseTrue)final_output.append(predicted_class[0][1]) return final_outputdef score(self, X_test, y_test):predictions self.predict(X_test)value 0for i in range(len(y_test)):if(predictions[i] y_test[i]):value 1return value / len(y_test)
2.1.3BM-FKNN
BM-FKNN是指一种基于贝叶斯模型的模糊K最近邻分类算法Bayesian Model-based Fuzzy K-Nearest Neighbor。BM-FKNN是对传统FKNN算法的改进它通过引入贝叶斯模型来提高分类性能。具体地说BM-FKNN使用贝叶斯分类器来计算每个类别的后验概率并将其作为FKNN的权重进而确定新数据点所属的类别。
BM-FKNN的主要优点是能够处理分类问题中的不确定性和噪声同时具有高效性和灵活性。与传统的K最近邻算法相比BM-FKNN能够更好地处理高维和大规模的数据集并且对于不平衡的数据集也表现良好。BM-FKNN在模式识别、图像处理、生物信息学和金融等领域有广泛的应用。
##BM-FKNNimport scipy.spatial
from collections import Counter
from operator import itemgetterclass BM_FKNN:def __init__(self, k):self.k kdef fit(self, X, y):self.X_train Xself.y_train ydef getDistance(self, X1, X2):return scipy.spatial.distance.euclidean(X1, X2)def getFirstDistance(self, X_train, X_test):distance []for i in range(len(X_train)):dist scipy.spatial.distance.euclidean(X_train[i] , X_test)distance.append([i, dist, self.y_train[i]])return distancedef nearestPoint(self, X_test):allPoint [ i for i in range(len(X_test))]distance self.getFirstDistance(self.X_train, X_test) distance_sorted sorted(distance, keyitemgetter(1))closest distance_sorted[0:self.k]closestPoint [ i[0] for i in closest]anothersPoint np.setdiff1d(allPoint, closestPoint)return closestPoint, anothersPointdef bonferroniMean(self, c, closestPoint, p, q): arrInner [self.X_train[e] for e in closestPoint if(self.y_train[e] ! c)] # j bukan angggota i arrOuter [self.X_train[q] for q in closestPoint if(self.y_train[q] c)]n len(closestPoint)if(n 1):inner [(sum(np.power(x, q)))/n for x in zip(*arrInner)]outer [(sum(np.power(x, p)))/(n-1) for x in zip(*arrOuter)]else:inner arrInner[0].copy()outer arrOuter[0].copy()Br [ np.power((inner[i]*outer[i]), (1/(pq)) ) for i in range(len(inner))]return Brdef fuzzy(self, arrBr, closestPoint, m):arrMembership []for localMean in arrBr:atas 0bawah 0for r in (closestPoint): if(localMean[1] self.y_train[r]):atas np.power((1/localMean[0]), (2/(m-1)))else:atas np.power((0/localMean[0]), (2/(m-1)))bawah np.power((1/localMean[0]), (2/(m-1)))arrMembership.append([atas/bawah, localMean[1]])return arrMembershipdef predict(self, X_test, p, q, m):final_output []for i in range(len(X_test)):localMean []closestPoint, anothersPoint self.nearestPoint(X_test[i])classes np.unique(self.y_train[closestPoint])if(len(classes) 1):final_output.append(classes[0]) else:arrBr []for j in classes:Br self.bonferroniMean(j, closestPoint, p, q)distBr self.getDistance(X_test[i], Br)arrBr.append([distBr, j])membership self.fuzzy(arrBr, closestPoint, m )predicted_class sorted(membership, keyitemgetter(0), reverseTrue)final_output.append(predicted_class[0][1])return final_outputdef score(self, X_test, y_test, p, q, m):predictions self.predict(X_test, p, q, m)value 0for i in range(len(y_test)):if(predictions[i] y_test[i]):value 1# print(value)return value / len(y_test) 2.1.3BM-FKNCN
一种基于局部Bonferroni均值的模糊K-最近质心近邻BM-FKNCN分类器该分类器根据最近的局部质心均值向量分配查询样本的类标签以更好地表示数据集的基础统计。由于最近中心邻域NCN概念还考虑了邻居的空间分布和对称位置因此所提出的分类器对异常值具有鲁棒性。此外所提出的分类器可以克服具有类不平衡的数据集中邻居的类支配因为它平均每个类的所有质心向量以充分解释类的分布。
##BM-FKNCNimport scipy.spatial
from collections import Counter
from operator import itemgetterclass BM_FKNCN:def __init__(self, k):self.k kdef fit(self, X, y):self.X_train Xself.y_train ydef getDistance(self, X1, X2):return scipy.spatial.distance.euclidean(X1, X2)def getFirstDistance(self, X_train, X_test):distance []for i in range(len(X_train)):dist scipy.spatial.distance.euclidean(X_train[i] , X_test)distance.append([i, dist, self.y_train[i]])return distancedef getCentroid(self, arrData):result[]dataTran np.array(arrData).Tfor i in range(len(dataTran)):result.append(np.mean(dataTran[i]))return resultdef kncn(self, X_test):myclass list(set(self.y_train))closestPoint []anothersPoint []for indexK in range(0, self.k):if(indexK 0):distance self.getFirstDistance(self.X_train, X_test) distance_sorted sorted(distance, keyitemgetter(1))closestPoint.append(distance_sorted[0][0])distance_sorted.pop(0)for anothers in (distance_sorted):anothersPoint.append(anothers[0]) else:arrDistance []closestPointTemp [self.X_train[r] for r in closestPoint]for r in (anothersPoint):arrQ closestPointTemp.copy()arrQ.append(self.X_train[r])arrDistance.append([r, self.getDistance(self.getCentroid(arrQ), X_test)])distance_sorted sorted(arrDistance, keyitemgetter(1))closestPoint.append(distance_sorted[0][0])anothersPoint np.setdiff1d(anothersPoint, closestPoint)return closestPoint, anothersPointdef bonferroniMean(self, c, closestPoint, p, q): arrInner [self.X_train[e] for e in closestPoint if(self.y_train[e] ! c)] # j bukan angggota i arrOuter [self.X_train[q] for q in closestPoint if(self.y_train[q] c)]n len(closestPoint)if(n 1):inner [(sum(np.power(x, q)))/n for x in zip(*arrInner)]outer [(sum(np.power(x, p)))/(n-1) for x in zip(*arrOuter)]else:inner arrInner[0].copy()outer arrOuter[0].copy()Br [ np.power((inner[i]*outer[i]), (1/(pq)) ) for i in range(len(inner))]return Brdef fuzzy(self, arrBr, closestPoint, m):arrMembership []for localMean in arrBr:atas 0bawah 0for r in (closestPoint):if(localMean[1] self.y_train[r]):atas np.power((1/localMean[0]), (2/(m-1)))else:atas np.power((0/localMean[0]), (2/(m-1)))bawah np.power((1/localMean[0]), (2/(m-1)))arrMembership.append([atas/bawah, localMean[1]])return arrMembershipdef predict(self, X_test, p, q, m):final_output []for i in range(len(X_test)):localMean []closestPoint, anothersPoint self.kncn(X_test[i])classes np.unique(self.y_train[closestPoint])if(len(classes) 1):final_output.append(classes[0]) else:arrBr []for j in classes:Br self.bonferroniMean(j, closestPoint, p, q)distBr self.getDistance(X_test[i], Br)arrBr.append([distBr, j])membership self.fuzzy(arrBr, closestPoint, m ) #Membership Degreepredicted_class sorted(membership, keyitemgetter(0), reverseTrue)final_output.append(predicted_class[0][1])return final_outputdef score(self, X_test, y_test, p, q, m):predictions self.predict(X_test, p, q, m)value 0for i in range(len(y_test)):if(predictions[i] y_test[i]):value 1return value / len(y_test)
2.2数据预处理
乳房X光检查数据集
train_path rdrive/MyDrive/BM-FKNCN-main/Dataset/mammographic_masses.xlsx
data_train pd.read_excel(train_path)
data_train.head() 输出数据集详细信息
data_train.info() 输出一个比重我不太清楚是什么应该是丢失数据集率
for col in data_train.columns:print(col, str(round(100* data_train[col].isnull().sum() / len(data_train), 2)) %) data_train.loc[(data_train[BI-RADS].isnull()True), BI-RADS] data_train[BI-RADS].mean()
data_train.loc[(data_train[Age].isnull()True), Age] data_train[Age].mean()
data_train.loc[(data_train[Shape].isnull()True), Shape] data_train[Shape].mean()
data_train.loc[(data_train[Margin].isnull()True), Margin] data_train[Margin].mean()
data_train.loc[(data_train[Density].isnull()True), Density] data_train[Density].mean()for col in data_train.columns:print(col, str(round(100* data_train[col].isnull().sum() / len(data_train), 2)) %) 2.3输出视图
import matplotlib.pyplot as plt
import seaborn as sns
import plotly.express as px
from matplotlib.colors import LinearSegmentedColormapcountClass data_train[Severity].value_counts().reset_index()
countClass.columns [Severity, count]
print(countClass)fig px.pie(countClass, valuescount, namesSeverity, titleClass Distribution, width700, height500
)fig.show()
统计了患病人数和比例 np.unique(np.array(data_train[Severity]))
症状这个变量矩阵肯定是0-1得与没得的差别 array([0, 1]) 输出患者和未患者的受每种影响因素及是否患病的两两关系图
features data_train.iloc[:,:5].columns.tolist()
plt.figure(figsize(18, 27))for i, col in enumerate(features):plt.subplot(6, 4, i*21)plt.subplots_adjust(hspace .25, wspace.3)plt.grid(True)plt.title(col)sns.kdeplot(data_train.loc[data_train[Severity]0, col], labelalive, color blue, shadeTrue, cut0)sns.kdeplot(data_train.loc[data_train[Severity]1, col], labeldead, color yellow, shadeTrue, cut0)plt.subplot(6, 4, i*22) sns.boxplot(y col, data data_train, xSeverity, palette [blue, yellow]) 数据处理
label_train data_train.iloc[:,-1].to_numpy()
fitur_train data_train.iloc[:,:5].to_numpy()
scaler MinMaxScaler(feature_range(0, 1))
scaler.fit(fitur_train)
fitur_train_normalize scaler.transform(fitur_train)
2.4调用各种方法看准确率
2.4.1BM-FKNCN
kf KFold(n_splits10, random_state1, shuffleTrue)
kf.get_n_splits(fitur_train_normalize)rataBMFKNCN[]
for train_index, test_index in kf.split(fitur_train_normalize):X_train, X_test fitur_train_normalize[train_index], fitur_train_normalize[test_index]y_train, y_test label_train[train_index], label_train[test_index]bmfkncn BM_FKNCN(9)bmfkncn.fit(X_train, y_train)prediction bmfkncn.score(X_test, y_test, 1, 1, 2)rataBMFKNCN.append(prediction)print(Mean Accuracy: , np.mean(rataBMFKNCN)) Mean Accuracy: 0.7960481099656358 (不知道为啥跑了8分钟……几个数为按理说不应该那么长
2.4.2BM-FKNN
kf KFold(n_splits10, random_state1, shuffleTrue)
kf.get_n_splits(fitur_train_normalize)rataBMFKNN []
for train_index, test_index in kf.split(fitur_train_normalize):X_train, X_test fitur_train_normalize[train_index], fitur_train_normalize[test_index]y_train, y_test label_train[train_index], label_train[test_index]bmfknn BM_FKNN(9)bmfknn.fit(X_train, y_train)prediction bmfknn.score(X_test, y_test, 1, 1, 2)rataBMFKNN.append(prediction)print(Mean Accuracy: , np.mean(rataBMFKNN)) Mean Accuracy: 0.7981421821305843 2.4.3FKNCN
kf KFold(n_splits10, random_state1, shuffleTrue)
kf.get_n_splits(fitur_train_normalize)rataFKNCN []
for train_index, test_index in kf.split(fitur_train_normalize):X_train, X_test fitur_train_normalize[train_index], fitur_train_normalize[test_index]y_train, y_test label_train[train_index], label_train[test_index]fkncn FKNCN(9)fkncn.fit(X_train, y_train)prediction fkncn.score(X_test, y_test)rataFKNCN.append(prediction)print(Mean Accuracy: , np.mean(rataFKNCN)) Mean Accuracy: 0.7783290378006873 这也跑了8分钟……看了FKNCN确实慢
2.4.4FKNN
kf KFold(n_splits10, random_state1, shuffleTrue)
kf.get_n_splits(fitur_train_normalize)accuracyFKNN []for train_index, test_index in kf.split(fitur_train_normalize):X_train, X_test fitur_train_normalize[train_index], fitur_train_normalize[test_index]y_train, y_test label_train[train_index], label_train[test_index]fknn FKNN(9)fknn.fit(X_train, y_train)prediction fknn.score(X_test, y_test)accuracyFKNN.append(prediction)print(Mean Accuracy: , np.mean(accuracyFKNN)) Mean Accuracy: 0.7783290378006873 2.4.5KNN
from sklearn.neighbors import KNeighborsClassifier
kf KFold(n_splits10, random_state1, shuffleTrue)
kf.get_n_splits(fitur_train_normalize)
rata []for train_index, test_index in kf.split(fitur_train_normalize):X_train, X_test fitur_train_normalize[train_index], fitur_train_normalize[test_index]y_train, y_test label_train[train_index], label_train[test_index]neigh KNeighborsClassifier(n_neighbors9)neigh.fit(X_train, y_train)prediction neigh.score(X_test, y_test)rata.append(prediction)print(Mean Accuracy: , np.mean(rata)) Mean Accuracy: 0.7981421821305843 柱状图看一下 简单看对于这组数据BM-FKNN和KNN表现最好。
来源有更多数据集供分析这里不再列举。
多组实验最后平均准确率 Average of Accuracy KNN 0,8630 FKNN 0,8666 FKNCN 0,8637 BM-FKNN 0,8634 BM-FKNCN 0,8986 实验结果表明与其他四个分类器相比BM-FKNCN实现了89.86%的最高总体平均分类精度。 来源GitHub - baguspurnama98/BM-FKNCN: A Bonferroni Mean Based Fuzzy K-Nearest Centroid Neighbor (BM-FKNCN), BM-FKNN, FKNCN, FKNN, KNN Classifier
