Introduction: The image classification problem of FAMNIST data collection consisting of five kinds of animals and five kinds of fruits was tested. This paper mainly focuses on the preparation of the previous database and the construction of the network. Detailed testing of the network can be found in Identification tests for ten species of animals and fruits in FAMNIST Test results in.
Keywords: FAMNIST, LENET, Artificial Neural Network
_01 Job preparation
1.1 Job Requirements
According to The fourth operation requirement of artificial neural network in 2021 Requirements for the first question: Design a convolution neural network (CNN) to complete the identification of a total of ten classes of objects for a given fruit (five classes) and animal (five classes).
1.1. 1 Database
The data is 16th Intelligent Visual Group Competition database in. Colored picture of five animals and five fruits, 283 in size × 283, stored in a directory:
├─FruitAnimal
- Animals
Animal
-Ox
-Dog
*Pig
- Cat
The horse
Fruit
Fruit
Durian
- Orange
- Apple
- Grapes
Banana
_Database can be used in AI Studio Database Download .
_02 Data Preprocessing
_Converts the original picture to a certain size of color, gray scale picture for network training.
- Processed picture download address: Propeller AI Studio - Artificial Intelligence Learning and Training Community
2.1 Data File Compilation
2.1.1 Studio Project
_Build training programs based on BML CodeLab working environment in AI Stdio, add " Five Animal and Fruit Databases "." Once in the environment, you can find the database in the environment:
data/data120001/FruitAnimal.zip
2.1. 2 Download data file
_It is difficult to operate in AI studio environment because the directory of the given data file is Chinese characters.
_Therefore, download the package locally and after decompression, change the names of all subdirectories to corresponding English. Then recompress the'/data'letter uploaded to AI Studio.
├─animal
│ ├─cat
│ ├─cow
│ ├─dog
│ ├─horse
│ └─pig
└─fruit
├─apple
├─banana
├─durian
├─grape
└─orange
2.1. 3 Compress files
_You can use the "Extract Compressed Files" right mouse button function in AI Studio for uploading the data file FruitAnimal. The zip file is decompressed. You can also use the Reading ZIP files in Python For compressed files.
2.1. 4 Collate into faMNIST database
There are 10 kinds of pictures (five animals, five fruits). Below we organize these pictures into different size picture databases.
(1) File Naming Specification
_I. Sorting Animal Fruits
_Ten animal and fruit species in the order defined in dict below:
afname = {'cat':0, 'cow':1, 'dog':2, 'horse':3, 'pig':4,
'apple':5, 'banana':6, 'durian':7, 'grape':8, 'orange':9}
_II. File name structure
_File names include: sort number, original file name information, as follows:
Sort_ 01.png
Following are some file names in the FAMNIST directory after processing:
0_01.png 1_39.png 2_67.png 4_03.png 5_45.png 6_87.png 8_49.png
0_02.png 1_3.png 2_68.png 4_04.png 5_46.png 6_88.png 8_50.png
0_03.png 1_40.png 2_69.png 4_05.png 5_47.png 6_89.png 8_51.png
0_04.png 1_41.png 2_70.png 4_06.png 5_48.png 6_90.png 8_52.png
0_05.png 1_42.png 2_71.png 4_07.png 5_49.png 6_91.png 8_53.png
0_06.png 1_43.png 2_72.png 4_08.png 5_50.png 6_92.png 8_54.png
0_07.png 1_44.png 2_73.png 4_09.png 5_51.png 6_93.png 8_55.png
0_08.png 1_45.png 2_74.png 4_10.png 5_52.png 7_01.png 8_56.png
0_09.png 1_46.png 2_75.png 4_11.png 5_53.png 7_02.png 8_57.png
0_10.png 1_47.png 2_76.png 4_12.png 5_54.png 7_03.png 8_58.png
0_11.png 1_48.png 2_77.png 4_13.png 5_55.png 7_04.png 8_59.png
0_12.png 1_49.png 2_78.png 4_14.png 5_56.png 7_05.png 8_60.png
0_13.png 1_4.png 2_79.png 4_15.png 5_57.png 7_06.png 8_61.png
0_14.png 1_50.png 2_80.png 4_16.png 5_58.png 7_07.png 8_62.png
0_15.png 1_51.png 2_81.png 4_17.png 5_59.png 7_08.png 8_63.png
0_16.png 1_52.png 2_82.png 4_18.png 5_60.png 7_09.png 8_64.png
0_17.png 1_53.png 2_83.png 4_19.png 5_61.png 7_10.png 8_65.png
_The number of each subclass of ten animal fruits is as follows:
1.cow:93
2.pig:88
3.cat:99
4.horse:95
5.dog:101
6.apple:88
7.durian:75
8.orange:86
9.grape:89
10.banana:93
All file: 907
- Number of files: 907
(2) Processing program
_Following is a program to copy and rename the original files:
import sys,os,math,time
import matplotlib.pyplot as plt
from numpy import *
import shutil
homepath = '/home/aistudio'
originpath = 'data/afmnist'
fapath = os.path.join(homepath, originpath)
def scanallpath(fpath):
subpath = []
fs1 = os.listdir(fpath)
for p in fs1[:2]:
scanfs = os.listdir(os.path.join(fpath,p))
sp = os.path.join(fpath, p)
for s in scanfs:
subpath.append(os.path.join(sp, s))
return subpath
allpath = scanallpath(fapath)
afname = {'cat':0, 'cow':1, 'dog':2, 'horse':3, 'pig':4,
'apple':5, 'banana':6, 'durian':7, 'grape':8, 'orange':9}
outpath = os.path.join(homepath, 'data/FAMNIST/FAMNIST-ALL')
allcount = 0
passid = 0
for p in allpath:
passid += 1
fdim = os.listdir(p)
pname = afname[p.split('/')[-1]]
subcount = 0
for fn in fdim:
if fn.find('png') < 0: continue
fname = os.path.join(p, fn)
newname = '%d_%s'%(pname, fn)
outname = os.path.join(outpath, newname)
shutil.copy(fname, outname)
allcount += 1
subcount += 1
print('{}.{}:{}'.format(passid, p.split('/')[-1], subcount))
print('All file: {}'.format(allcount))
2.2 FAMNIST database
_Organize animal and fruit data collections for experiments in the field:
- Resize pictures into small pictures of different sizes;
- Enhance the picture;
2.2. 1 Original Gallery
_This was the original picture database.
Database parameters: Categories: 10 categoriesColor: Color Pictures
Size: 283 × 283
#!/usr/local/bin/python
# -*- coding: gbk -*-
#============================================================
# SHOWPIC.PY -- by Dr. ZhuoQing 2021-12-16
#
# Show some pictures in the FAMNIST directory in shuffle order.
#
# Usage : showpic directory
#
# Note:
#============================================================
from headm import * # =*
import cv2
famnist = '/home/aistudio/data/famnist'
dirstr = 'famnist-all'
#------------------------------------------------------------
dirall = os.path.join(famnist, dirstr)
filedim = os.listdir(dirall)
random.shuffle(filedim)
#------------------------------------------------------------
ROW_NUM = 3
COL_NUM = 5
plt.figure(figsize=(10,6))
for j in range(ROW_NUM):
for i in range(COL_NUM):
id = j*COL_NUM + i
fn = os.path.join(dirall, filedim[id])
img = cv2.imread(fn).T[::-1].T
plt.subplot(ROW_NUM, COL_NUM, id+1)
plt.imshow(img)
plt.axis('off')
plt.title(filedim[id])
plt.show
#------------------------------------------------------------
# END OF FILE : SHOWPIC.PY
#============================================================
2.2.2 FAMNIST10
_This is to change the original picture to 32 × 32-size color picture.
Photo Gallery Parameters: Storage directory: FAMNIST10Color: Color Pictures
Size: 32 × 32
(1) Converter
import sys,os,math,time
import matplotlib.pyplot as plt
from numpy import *
import cv2
famnist = '/home/aistudio/data/famnist'
alldir = 'famnist-all'
outdir = 'famnist10'
allpath = os.path.join(famnist, alldir)
filedim = os.listdir(allpath)
outdir = os.path.join(famnist, outdir)
imgWidth = 32
imgHeight = 32
for id, f in enumerate(filedim):
infile = os.path.join(allpath, f)
outfile = os.path.join(outdir, f)
img = cv2.imread(infile)
imgsize = cv2.resize(img, (imgWidth, imgHeight), cv2.INTER_LINEAR)
cv2.imwrite(outfile, imgsize)
print("Process %d, %s"%(id, f))
(2) Picture samples
2.2.3 FAMNIST10-gray
_This is to convert the FAMNIST10 above into a grayscale picture.
Photo Gallery Parameters: Storage directory: FAMNIST10-grayColor: Grayscale Picture
Size: 32 × 32
imgWidth = 32
imgHeight = 32
for id, f in enumerate(filedim):
infile = os.path.join(allpath, f)
outfile = os.path.join(outdir, f)
img = cv2.imread(infile, cv2.IMREAD_GRAYSCALE)
imgsize = cv2.resize(img, (imgWidth, imgHeight), cv2.INTER_LINEAR)
2.2.4 FAMNIST10-64
_This is the size 64 × 64 picture library.
Photo Gallery Parameters: Storage directory: famnist10-64Color: Color Pictures
Size: 64 × 64
2.2.5 FAMNIST10-64-gray
_This is the famnist10-64 picture above stored as a grayscale picture.
Photo Gallery Parameters:
: Storage directory: famnist10-64-gray
Color: Grayscale Picture
Size: 64 × 64
_03 Build CNN
3.1 LeNet Recognition
3.1. 1 Build training data
_Reads picture data from a storage directory into memory for training on the network.
(1) Read in picture data and labels
_According to the definition of file name in [2.1.4.1: File Naming Specification], the various labels of pictures can also be obtained during the reading of picture data.
_I. Reader
import sys,os,math,time
import matplotlib.pyplot as plt
from numpy import *
import cv2
#------------------------------------------------------------
famnist = '/home/aistudio/data/famnist'
imgdir = 'famnist10-gray'
#------------------------------------------------------------
def loadimgdata(imgdir):
'''
loadimgdata: Load test image data into RAM
Param imgdir: Directory for storing the image picture .
Return: imgdata,imglabel
'''
imgfile = os.listdir(imgdir)
imgdata = []
imglabel = []
for f in imgfile:
img = cv2.imread(os.path.join(imgdir, f))
imgdata.append(img.T)
imglabel.append(int(f[:1]))
return array(imgdata), array(imglabel)
imgdata,imglabel = loadimgdata(os.path.join(famnist,imgdir))
_II. Data Results
print(imgdata.shape) print(imglabel)
(906, 3, 32, 32) [6 6 3 4 9 6 0 6 3 2 2 3 2 0 4 3 8 9 2 9 9 9 0 4 0 3 9 4 1 3 8 4 0 4 2 8 6 4 9 0 0 4 9 2 3 7 8 7 9 5 6 0 7 2 3 6 5 3 0 9 8 6 0 0 3 0 4 0 9 4 0 4 3 6 0 5 7 9 3 3 1 8 2 4 6 4 6 5 2 9 9 2 3 0 5 8 3 7 6 0 2 4 5 7 8 5 6 9 3 4 8 4 8 1 8 9 2 4 8 7 3 2 6 8 9 0 5 1 8 3 5 4 3 6 8 1 6 9 0 0 9 0 5 5 8 9 9 7 7 5 4 7 1 3 8 9 5 2 2 6 6 1 6 6 7 7 6 4 4 5 4 0 1 3 5 3 6 9 1 6 2 0 6 2 3 9 8 2 8 5 5 9 5 2 0 2 4 1 5 6 1 7 2 0 0 5 7 8 1 4 0 1 3 4 9 1 6 8 8 9 1 9 9 8 2 4 8 3 9 6 7 6 2 6 5 4 4 0 4 2 3 3 3 1 3 1 1 8 0 2 8 4 8 3 3 6 9 0 8 7 2 7 7 9 0 3 2 2 3 2 2 0 4 5 5 3 7 0 7 0 9 4 4 5 6 2 2 0 9 7 3 6 4 0 4 1 0 0 1 2 5 6 6 3 5 6 6 1 3 0 5 3 2 6 9 2 0 7 5 2 2 0 4 3 5 4 7 0 5 2 1 3 4 2 5 2 9 4 9 4 2 1 7 6 2 5 1 0 6 0 0 4 9 6 4 7 3 4 9 4 8 2 7 8 2 3 8 6 6 6 4 0 3 3 7 8 8 4 9 6 9 7 3 8 5 2 1 8 8 4 9 2 0 8 6 5 1 4 8 1 7 6 1 5 8 8 0 0 0 1 1 3 5 0 8 3 3 6 8 7 6 7 4 8 6 7 8 7 3 0 3 1 8 5 5 4 2 8 4 0 2 9 6 2 0 2 1 7 4 8 5 1 4 2 6 9 3 1 1 2 1 1 1 5 4 9 5 0 1 7 5 8 0 3 3 0 2 1 5 7 1 7 0 8 3 9 6 8 0 2 9 8 1 6 3 7 7 2 3 3 1 2 2 9 5 4 3 9 3 1 5 2 6 2 5 2 8 0 3 9 7 1 7 7 7 7 4 6 1 3 9 4 0 3 7 0 5 8 8 9 7 1 1 9 8 1 1 0 7 3 8 3 2 0 9 1 7 7 8 4 1 1 0 8 2 5 5 2 7 1 9 2 8 6 1 9 6 6 6 9 2 5 4 1 9 5 2 3 8 6 5 2 8 9 1 6 6 5 0 2 5 7 1 6 0 1 6 7 6 5 8 0 9 6 0 3 0 4 2 9 3 1 2 7 3 6 1 4 3 4 7 2 6 5 4 8 0 6 2 5 8 8 2 6 1 8 0 4 6 4 9 0 4 8 7 1 6 9 4 8 1 3 5 5 5 1 2 9 4 2 1 5 6 4 0 0 3 1 1 9 8 4 1 2 7 2 5 5 3 7 6 0 4 0 1 0 7 1 4 5 3 8 6 5 9 1 6 4 7 1 9 7 3 1 7 5 7 7 1 5 6 9 2 6 5 7 3 9 3 5 5 4 8 5 6 9 0 3 2 8 0 1 0 6 9 2 4 2 3 8 4 0 5 8 0 7 9 5 9 0 6 7 4 2 6 6 9 1 8 1 0 7 0 9 9 2 3 0 2 3 9 8 8 3 3 3 8 7 5 2 8 2 1 0 2 2 0 9 3 3 0 1 9 8 2 0 4 1 6 5 3 6 0 8 3 9 5 1 4 1 8 5 2 2 8 1 1 1 4 6 0 4 8 5 4 0 6 2 1 5 6 0 2 5 7 5 6 8 9 5 0 0 1 8 4 1 3 4 1 7 8 6 5 7 9 4 5 3 3 3 2 9 6 3 5 3 5 5 2 7 0 1 2 9 2 6 9 4 7 0 2 8 6 9 4 8 0 7 7 2 2 4 3 2 5 1]
_During data reading, the data is converted into RGB hierarchical data structure through matrix transpose operation.
(2) Build Dataset
class famnist(paddle.io.Dataset):
def __init__(self, num_samples):
super(famnist, self).__init__()
self.num_samples = num_samples
def __getitem__(self, index):
data = TT(imgdata[index].astype('float64'))
label = TT(imglabel[index].astype('int64'))
return data, label
def __len__(self):
return self.num_samples
_dataset = famnist(800)
train_loader = paddle.io.DataLoader(_dataset, batch_size=20, shuffle=True)
_I. Test Dataloader
_to train_loader obtains the structure of the data and displays pictures to verify that the above programs are working properly.
data = train_loader().next() print(data)
[Tensor(shape=[20, 3, 32, 32], dtype=float64, place=CPUPlace, stop_gradient=True,
[[[[69. , 19. , 79. , ..., 178., 140., 173.],
[16. , 13. , 54. , ..., 214., 194., 153.],
[30. , 29. , 28. , ..., 201., 179., 191.],
...,
[254., 254., 254., ..., 254., 254., 254.],
[253., 253., 253., ..., 253., 253., 253.],
[253., 253., 253., ..., 253., 253., 253.]]]]), Tensor(shape=[20, 1], dtype=int64, place=CPUPlace, stop_gradient=True,
[[1],
[6],
[9],
[7],
[0],
[4],
[5],
[4],
[5],
[9],
[5],
[3],
[0],
[4],
[6],
[0],
[9],
[9],
[8],
[0]])]
_Use graphics to show train_loader gets the picture.
data = train_loader().next()
ROW_NUM = 3
COL_NUM = 5
plt.figure(figsize=(10,6))
for j in range(ROW_NUM):
for i in range(COL_NUM):
id = j*COL_NUM + i
img = data[0].numpy()[id].T
print(img.shape)
plt.subplot(ROW_NUM, COL_NUM, id+1)
plt.imshow(img, cmap='gray')
plt.axis('off')
_From the above results, you can see that the training data loading function is constructed correctly.
3.1. 2 Build LeNet Network
imageSize = 64
ks = 5
L = ((imageSize-ks+1)//2-ks+1)//2
class mnist(paddle.nn.Layer):
def __init__(self, ):
super(mnist, self).__init__()
self.conv1 = paddle.nn.Conv2D(in_channels=1, out_channels=6, kernel_size=ks, stride=1, padding=0)
self.conv2 = paddle.nn.Conv2D(in_channels=6, out_channels=16, kernel_size=ks, stride=1, padding=0)
self.mp1 = paddle.nn.MaxPool2D(kernel_size=2, stride=2)
self.mp2 = paddle.nn.MaxPool2D(kernel_size=2, stride=2)
self.L1 = paddle.nn.Linear(in_features=16*L*L, out_features=120)
self.L2 = paddle.nn.Linear(in_features=120, out_features=86)
self.L3 = paddle.nn.Linear(in_features=86, out_features=10)
def forward(self, x):
x = self.conv1(x)
x = F.relu(x)
x = self.mp1(x)
x = self.conv2(x)
x = F.relu(x)
x = self.mp2(x)
x = paddle.flatten(x, start_axis=1, stop_axis=-1)
x = self.L1(x)
x = F.relu(x)
x = self.L2(x)
x = F.relu(x)
x = self.L3(x)
return x
3.1. 3 Training LeNet Network
net = mnist()
EPOCH_NUM = 100
optimizer = paddle.optimizer.Adam(learning_rate=0.001, parameters=net.parameters())
for epoch in range(EPOCH_NUM):
for batchid, data in enumerate(train_loader()):
out = net(data[0])
loss = F.cross_entropy(out, data[1])
acc = paddle.metric.accuracy(out, data[1])
loss.backward()
optimizer.step()
optimizer.clear_grad()
if batchid %100==0:
print("Pass:{}, Loss:{}, Acc:{}".format(epoch,loss.numpy(), acc.numpy()))
from headm import *
strid = 12
tspgetdopstring(-strid)
strall = clipboard.paste().split('\r\n')
accdim = []
for s in strall:
ss = s.split(',')
acc = ss[-1].replace('Acc:[','').replace(']','')
accdim.append(float(acc.strip(' ')))
printf(accdim)
plt.plot(accdim)
plt.xlabel("Step")
plt.ylabel("Accuracy")
plt.grid(True)
plt.title("Train Accuracy")
plt.tight_layout()
plt.show()
Summary of 04 Operation
_The image classification problem of FAMNIST data collection consisting of five animals and five fruits was tested. This paper mainly focuses on the preparation of the previous database and the construction of the network. Detailed testing of the network can be found in Identification tests for ten species of animals and fruits in FAMNIST Test results in.
4.1 Program Code
4.1.1 LENET Program
#!/usr/local/bin/python
# -*- coding: gbk -*-
#============================================================
# LENET.PY -- by Dr. ZhuoQing 2021-12-16
#
# Note:
#============================================================
from headm import * # =
import cv2
import paddle
import paddle.nn.functional as F
from paddle import to_tensor as TT
from paddle.nn.functional import square_error_cost as sqrc
#------------------------------------------------------------
famnist = '/home/aistudio/data/famnist'
imgdir = 'famnist10-64-gray'
#------------------------------------------------------------
def loadimgdata(imgdir):
'''
loadimgdata: Load test image data into RAM
Param imgdir: Directory for storing the image picture .
Return: imgdata,imglabel
'''
imgfile = os.listdir(imgdir)
imgdata = []
imglabel = []
for f in imgfile:
img = cv2.imread(os.path.join(imgdir, f))
imgdata.append(img.T[0][newaxis,:])
imglabel.append(int(f[:1]))
return array(imgdata), array(imglabel)
imgdata,imglabel = loadimgdata(os.path.join(famnist,imgdir))
print(len(imglabel))
print(imgdata.shape)
#------------------------------------------------------------
'''
printf(imgdata.shape)
printf(imglabel)
'''
#------------------------------------------------------------
class famnist(paddle.io.Dataset):
def __init__(self, num_samples):
super(famnist, self).__init__()
self.num_samples = num_samples
def __getitem__(self, index):
data = imgdata[index]/255
label = imglabel[index]
return TT(data, dtype='float32'), TT(label, dtype='int64')
def __len__(self):
return self.num_samples
_dataset = famnist(800)
train_loader = paddle.io.DataLoader(_dataset, batch_size=100, shuffle=True)
#------------------------------------------------------------
#data = train_loader().next()
#print(data[0].numpy())
'''
ROW_NUM = 3
COL_NUM = 5
plt.figure(figsize=(10,6))
for j in range(ROW_NUM):
for i in range(COL_NUM):
id = j*COL_NUM + i
img = data[0].numpy()[id][0].T
# print(img.shape)
plt.subplot(ROW_NUM, COL_NUM, id+1)
plt.imshow(img, cmap='gray')
plt.axis('off')
'''
#------------------------------------------------------------
imageSize = 64
ks = 5
L = ((imageSize-ks+1)//2-ks+1)//2
class mnist(paddle.nn.Layer):
def __init__(self, ):
super(mnist, self).__init__()
self.conv1 = paddle.nn.Conv2D(in_channels=1, out_channels=6, kernel_size=ks, stride=1, padding=0)
self.conv2 = paddle.nn.Conv2D(in_channels=6, out_channels=16, kernel_size=ks, stride=1, padding=0)
self.mp1 = paddle.nn.MaxPool2D(kernel_size=2, stride=2)
self.mp2 = paddle.nn.MaxPool2D(kernel_size=2, stride=2)
self.L1 = paddle.nn.Linear(in_features=16*L*L, out_features=120)
self.L2 = paddle.nn.Linear(in_features=120, out_features=86)
self.L3 = paddle.nn.Linear(in_features=86, out_features=10)
def forward(self, x):
x = self.conv1(x)
x = F.relu(x)
x = self.mp1(x)
x = self.conv2(x)
x = F.relu(x)
x = self.mp2(x)
x = paddle.flatten(x, start_axis=1, stop_axis=-1)
x = self.L1(x)
x = F.relu(x)
x = self.L2(x)
x = F.relu(x)
x = self.L3(x)
return x
#------------------------------------------------------------
net = mnist()
EPOCH_NUM = 100
optimizer = paddle.optimizer.Adam(learning_rate=0.001, parameters=net.parameters())
for epoch in range(EPOCH_NUM):
for batchid, data in enumerate(train_loader()):
out = net(data[0])
loss = F.cross_entropy(out, data[1])
acc = paddle.metric.accuracy(out, data[1])
loss.backward()
optimizer.step()
optimizer.clear_grad()
if batchid %100==0:
print("Pass:{}, Loss:{}, Acc:{}".format(epoch,loss.numpy(), acc.numpy()))
#------------------------------------------------------------
# END OF FILE : LENET.PY
#============================================================
4.1. 2 Picture Processor
#!/usr/local/bin/python
# -*- coding: gbk -*-
#============================================================
# FAMNIST10.PY -- by Dr. ZhuoQing 2021-12-16
#
# Note:
#============================================================
from headm import * # =*
import cv2
famnist = '/home/aistudio/data/famnist'
alldir = 'famnist-all'
outdir = 'famnist10-64-gray'
#------------------------------------------------------------
allpath = os.path.join(famnist, alldir)
filedim = os.listdir(allpath)
#print(filedim)
#------------------------------------------------------------
outdir = os.path.join(famnist, outdir)
imgWidth = 64
imgHeight = 64
for id, f in enumerate(filedim):
infile = os.path.join(allpath, f)
outfile = os.path.join(outdir, f)
img = cv2.imread(infile, cv2.IMREAD_GRAYSCALE)
# img = cv2.imread(infile, cv2.IMREAD_COLOR)
imgsize = cv2.resize(img, (imgWidth, imgHeight), cv2.INTER_LINEAR)
# printf(imgsize.shape)
# break
cv2.imwrite(outfile, imgsize)
print("Process %d, %s"%(id, f))
#------------------------------------------------------------
# END OF FILE : FAMNIST10.PY
#============================================================
4.1. 3 Display catalog photos
#!/usr/local/bin/python
# -*- coding: gbk -*-
#============================================================
# SHOWPIC.PY -- by Dr. ZhuoQing 2021-12-16
#
# Show some pictures in the FAMNIST directory in shuffle order.
#
# Usage : showpic directory
#
# Note:
#============================================================
from headm import * # =*
import cv2
famnist = '/home/aistudio/data/famnist'
dirstr = 'famnist10-64-gray'
#------------------------------------------------------------
dirall = os.path.join(famnist, dirstr)
filedim = os.listdir(dirall)
random.shuffle(filedim)
#------------------------------------------------------------
ROW_NUM = 3
COL_NUM = 5
plt.figure(figsize=(10,6))
for j in range(ROW_NUM):
for i in range(COL_NUM):
id = j*COL_NUM + i
fn = os.path.join(dirall, filedim[id])
img = cv2.imread(fn).T[::-1].T
plt.subplot(ROW_NUM, COL_NUM, id+1)
plt.imshow(img)
plt.axis('off')
plt.title(filedim[id])
plt.show
#------------------------------------------------------------
# END OF FILE : SHOWPIC.PY
#============================================================
4.1. 4 Generate the original picture collection
#!/usr/local/bin/python
# -*- coding: gbk -*-
#============================================================
# FAMNIST.PY -- by Dr. ZhuoQing 2021-12-16
#
# Note:
#============================================================
from headm import * # =*
import shutil
homepath = '/home/aistudio'
originpath = 'data/afmnist'
fapath = os.path.join(homepath, originpath)
#------------------------------------------------------------
def scanallpath(fpath):
subpath = []
fs1 = os.listdir(fpath)
for p in fs1[:2]:
scanfs = os.listdir(os.path.join(fpath,p))
sp = os.path.join(fpath, p)
for s in scanfs:
subpath.append(os.path.join(sp, s))
return subpath
#------------------------------------------------------------
allpath = scanallpath(fapath)
#printf(allpath)
#------------------------------------------------------------
afname = {'cat':0, 'cow':1, 'dog':2, 'horse':3, 'pig':4,
'apple':5, 'banana':6, 'durian':7, 'grape':8, 'orange':9}
outpath = os.path.join(homepath, 'data/FAMNIST/FAMNIST-ALL')
allcount = 0
passid = 0
for p in allpath:
passid += 1
fdim = os.listdir(p)
pname = afname[p.split('/')[-1]]
subcount = 0
for fn in fdim:
if fn.find('png') < 0: continue
fname = os.path.join(p, fn)
newname = '%d_%s'%(pname, fn)
outname = os.path.join(outpath, newname)
shutil.copy(fname, outname)
allcount += 1
subcount += 1
# printf(outname)
printf('{}.{}:{}'.format(passid, p.split('/')[-1], subcount))
printf('All file: {}'.format(allcount))
#------------------------------------------------------------
# END OF FILE : FAMNIST.PY
#============================================================
Links to related literature:
- The fourth operation requirement of artificial neural network in 2021
- Rules for the 17th Smart Car Competition Intelligent Visual Group
- AI Studio Database Download
- Propeller AI Studio - Artificial Intelligence Learning and Training Community
- Reading ZIP files in Python
- 404
Related chart links:
- Figure 1.1. 1 Database in AI Studio
- Figure 2.1. 1 Establish AI Studio Project
- Figure 2.1. 2 The directory in the original compression package is Chinese characters
- Figure 2.2. 1 Typical items are centralized in raw data
- Figure 2.2.2 FAMNIST10 32 × 32 color pictures
- Figure 2.2.3 famnist10-gray database
- Figure 2.2.4 famnist10-64 Picture Sample
- Figure 2.2.5 famnist10-64-gray
- Figure 3.1. 1 Picture data from Database Loader
- Figure 3.1. 2 Precision change curve during training
- Figure 4.1 Classification tests for always animals and fruits in FAMNIST