Build fat environment from the new computer, with pictures and texts

Note: This article is very long and should be read. Reprinting without permission is strictly prohibited! Please obtain the permission of the author before reprinting

Write in front

During the holiday, I was doing a service outsourcing competition, which was related to federal learning, so I wrote this blog (Series) to record my learning.
Thanks to my senior zjj, most of the content of this blog is also from his hands. He led a real Xiaobai, and now a Xiaobai who knows a little. Here is a link to his csdn homepage (but I don't seem to know 55 yet. I'll add it here when I get there!)

No more nonsense, let's start now!

Download and install VMware WorkStation 16

Run the file vmware-workstation-full-16.0 Exe and complete the operation according to the prompt.
Key: (I won't provide this, dddd, search by yourself)

Install CentOS7 (those already installed can be omitted. This part is more detailed)

CentOS 7.0 is installed here Version of 9

  1. Click "create new virtual machine" on the home page. You can also click file - > new virtual machine

  2. Select "customize (Advanced)" and click "next"

  3. By default, click "next"

  4. Select "install the operating system later" and then click "next"

  5. Select Linux as the operating system of the client, CentOS7 64 bit as the version, and then next

  6. Name the virtual machine, set the save path, and then go to the next step

  7. Configure the processor. I've tried a single processor. The system is too slow. I suggest more. Next step

  8. Allocate memory, which is similar to cpu. If there are fewer cards, go to the next step

  9. Generally, NAT is selected. It is OK by default. Next step

  10. Next, default all the way until the disk capacity is specified. Give multiple points and reserve space for fat. Next

  11. Always default to the next step, and finally click finish.

  12. In the virtual machine, click virtual machine – > settings – > CD / DVD (IDE), select your own image file, and select centos-7-x86 here_ 64-DVD-2009. ISO is enough

  13. Start the virtual machine, enter the system installation, press enter and wait

  14. Easy to learn and understand, choose Chinese

  15. Select keyboard and add English

  16. After the disk partition is clicked in, it is completed directly

  17. Select "network and host name" and open the network

  18. Select "software selection" to install the software

  19. Click Start installation

  20. Set the password and create the user. The user can not create it

  21. Wait. When it's finished, restart it

  22. Just go in with permission, click agree and finish

  23. Then move on until you choose the region and Shanghai

  24. Then move forward, skip, etc. write in English in "about you" and move forward

  25. Password setting, forward

  26. Installation of CentOS7 system completed

Install python3 (note that the network should remain connected)

  1. Right click the desktop to open the terminal
  2. Use root privileges
  1. Install Python 3.0 Some dependent packages of 6
rm -f /var/run/
yum install openssl-devel bzip2-devel expat-devel gdbm-devel readline-devel sqlite-devel

The results shown in the figure below indicate success

  1. Download Python 3 six point five

  1. Extract to the specified folder
tar -xzvf Python-3.6.5.tgz -C /usr/local

  1. Enter the extracted file
cd /usr/local/Python-3.6.5

  1. Install gcc using yum or up2date
yum install gcc

  1. compile
./configure prefix=/usr/local/python3
make && make install

  1. Configure soft links
cd /usr/bin
mv python python.backup    
ln -s /usr/local/python3/bin/python3.6 /usr/bin/python
ln -s /usr/local/python3/bin/python3.6 /usr/bin/python3

  1. Change Python in the first line of the following three files to python2
    After entering the file, enter i to indicate insertion, and the bottom will become INSERT, which can be modified at this time.
    After modifying the contents of the file, save it and press Esc+:+wq to save it
vim /usr/bin/yum
vim /usr/libexec/urlgrabber-ext-down
vim /usr/bin/yum-config-manager

Install docker

  1. Install the package using yum or up2date
yum install -y yum-utils device-mapper-persistent-data lvm2

  1. Install docker engine community
    Use Docker warehouse for installation:
    Before installing Docker engine community on the new host for the first time, you need to set up the Docker warehouse.
    After that, Docker can be installed and updated from the warehouse.
yum install -y yum-utils \
 device-mapper-persistent-data \
  1. Set up a stable warehouse
yum-config-manager --add-repo

  1. To install docker engine community:
    To install a specific version of docker engine community, list the available versions in the repository, and then select and install them. The steps are as follows:
Lists and sorts the versions available in your repository. This example sorts the results by version number (from high to low):
yum list docker-ce --showduplicates | sort -r
Install a specific version with its full package name, which is the package name( docker-ce)Add the version string (second column) from the first colon(:)Continue to the first hyphen and use hyphens(-)separate. For example: docker-ce-18.09.1: 
yum install docker-ce-18.09.0 docker-ce-cli-18.09.0
Start and join the startup self start:
systemctl start docker
systemctl enable docker

By running hello-world Image to verify that it is installed correctly Docker Engine-Community 
docker run hello-world
Check version
docker -v
docker version

  1. Install docker compose
curl -L "$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

  1. View version
chmod +x /usr/local/bin/docker-compose
docker-compose version

Docker command

docker imagesDisplay mirror
docker ps -aView the list of containers in the current system.
docker ps -lThe containers that were last run are listed, including those that are running and those that have been stopped.
docker startStart one or more containers that have been stopped
docker stopStop a running container
docker rmDelete one or more containers
docker execExecute commands in a running container
docker restartRestart container
docker stop [container name / container ID]Stop container operation

Download fat project

  1. Download:

Or the docker_standalone_fate_1.6.0.tar.gz import directly from this machine (recommended)

  1. Decompression:
tar -xzvf docker_standalone_fate_1.6.0.tar.gz

  1. Deployment:
cd docker_standalone_fate_1.6.0

  1. Unit test:
CONTAINER_ID=`docker ps -aqf "name=fate"`
docker exec -t -i ${CONTAINER_ID} bash
bash ./python/federatedml/test/
  1. Toy test:
CONTAINER_ID=`docker ps -aqf "name=fate"`
docker exec -t -i ${CONTAINER_ID} bash
python ./examples/toy_example/ 10000 10000 0

Fate command

  1. View Table
python python/fate_flow/ -f table_info -n experiment -t breast_homo_host
  1. Delete table
python -f table_delete -n $namespace -t $table_name
python -f table_delete -j $job_id
python python/fate_flow/ -f table_delete -n experiment -t breast_homo_host

Horizontal Federation learning example

  1. Upload training data
python python/fate_flow/ -f upload -c  /fate/examples/dsl/v1/homo_logistic_regression/upload_data_host.json 
python python/fate_flow/ -f upload -c  /fate/examples/dsl/v1/homo_logistic_regression/upload_data_guest.json
  1. Submit training tasks
python  python/fate_flow/ -f submit_job -d /fate/examples/dsl/v1/homo_logistic_regression/test_homolr_train_job_dsl.json  -c /fate/examples/dsl/v1/homo_logistic_regression/test_homolr_train_job_conf.json
  1. Forecast data
python python/fate_flow/ -f submit_job -c /fate/examples/dsl/v1/homo_logistic_regression/test_predict_conf.json
 Remember to modify test_predict_conf.json In the file model_version: {jobid}

Horizontal Federation learning example

  1. Upload training data
python python/fate_flow/ -f upload -c  /fate/examples/dsl/v1/homo_logistic_regression/up_dt_guest.json -drop 1
 python python/fate_flow/ -f upload -c  /fate/examples/dsl/v1/homo_logistic_regression/up_dt_host.json -drop 1
  1. Training tasks
python  python/fate_flow/ -f submit_job -d /fate/examples/dsl/v1/hetero_logistic_regression/test_hetero_lr_train_job_dsl.json  -c /fate/examples/dsl/v1/hetero_logistic_regression/test_hetero_lr_train_job_conf.json
  1. Training tasks
python python/fate_flow/ -f submit_job -c /fate/examples/dsl/v1/hetero_logistic_regression/test_predict_conf.json
 Remember to modify test_predict_conf.json In the file model_version: {jobid}

Write at the end

Special thanks

When I really typed down the full text, I realized how hard it was for zjj senior students to make tutorials for us at that time. It was too tired!
The tutorial of senior students is presented in the form of word. In order to show it more intuitively and make it more convenient for them to see, I wrote this blog.


If you like this article, you might as well like it
Any questions are welcome to be raised in the comment area and discussed together for common progress!
More fat learning progress will be continuously updated in the future

Keywords: Python Machine Learning FATE

Added by Naez on Thu, 03 Feb 2022 10:19:24 +0200