Big data: visualization of Douban TV series crawler anti climbing agent IP, spark cleaning and flash framework
Full steps of Douban movie big data project
1. Douban reptile:
When I started to write Douban TV series crawler, I thought it was very simple, but in practice, there was an IP sealing situation, which led to my distress for a long time, and now I finally wrote it
No more nonsense, just go to the code:
The run function is ...
Added by gregor171 on Fri, 04 Mar 2022 14:51:48 +0200
python tool function code
1. Implement a decorator that determines the presence of the redis server
from functools import wraps
from flask import g
from rmon.common.rest import RestException
class ObjectMustBeExist:
"""The ornament is used for a Server Instance deletion check to ensure operation object must exist
"""
def __init__(self, object_class):
...
Added by coho75 on Mon, 28 Feb 2022 19:28:57 +0200
[python] Flash Web Application Framework
brief introduction
Flask is a lightweight WSGI Web application framework. It is designed to make getting started quick and easy and can be extended to complex applications. Originally a simple wrapper around Werkzeug and Jinja, it has become one of the most popular Python Web application frameworks.
Flash source code Flash User Guide
1, ...
Added by joshi_v on Sun, 27 Feb 2022 11:54:07 +0200
[customized project] [M13] [data monitoring + data analysis + large visual screen] - Key Technology Python Flash + pandas + ecarts + Excel
catalogue
1, Project requirements
2, Project architecture
3, Project source code analysis
1. http server: Python Flask framework;
http code
Interface resolution
2. File monitoring: Python win32con module;
Monitoring code
Code parsing
3. Data analysis: Python Pandas library;
Pandas Series code
File example
4. Data visualization: ech ...
Added by miracle_potential on Sun, 20 Feb 2022 18:23:55 +0200
[Flask] create ORM mapping and parameter interpretation
Create a class. A class corresponds to a table in a database, and the data attributes of the class correspond to the field names in the table. This class is called mapping class.
1, Proactively create mappings
Classes mapped using the Declarative system are defined according to the base class. In other words, each mapped class needs to inhe ...
Added by jbalanski on Thu, 10 Feb 2022 07:48:36 +0200
Parameter verification of flash development skills wtforms wtforms JSON
Parameter verification of flash development skills
I usually use or learn some flash development skills in development. I need to have a solid foundation of flash.
1. Request parameter classification
Generally speaking, the request sent by the front end roughly includes the following three types of parameters: url path parameters, url query ...
Added by monkeybidz on Wed, 09 Feb 2022 17:11:20 +0200
Steps for deploying flash with DockerDesktop installed on Mac
1, Download docker
stay Docker official website Find the corresponding mac download in. After clicking in, I choose the version of the corresponding chip of my computer and start downloading. I own m1 chip.
2, Install and run docker on MAC computer
1. Double click Docker DMG open the installer and drag the Docker icon to the Applicatio ...
Added by LuciBKen on Sun, 30 Jan 2022 20:22:28 +0200
Brother ape takes you to the flash framework
1, Flask Foundation
[example 1] write the first flash program, and the code is as follows:
from flask import Flask # 1. Import Flask class
"""
2.Create an instance of this class. The first parameter is the name of the application module or package. If a single module is used(Such place),Should be used__name__.
The name of the module w ...
Added by acidbox on Sun, 30 Jan 2022 04:07:29 +0200
Constructing pure RSS official account information flow based on Liuli
First of all, what is Liuli? This is an open source project I recently developed. The main purpose is to enable friends with reading habits to quickly build a multi-source, clean and personalized reading environment.Why Liuli?Liuli was originally named 2C, and the friends of the communication group behind provided the name of Liuli, which is ta ...
Added by phpcip28 on Fri, 28 Jan 2022 00:02:52 +0200
gevent file read / write pit
Scenarios and phenomena to be optimized
The project gives the flash gevent framework, which will periodically (5 seconds) write a json file to the file system
The file size is 40M
The phenomenon is that gevent will not switch the code of other processes until the file is written
principle
When reading and writing files, linux will block ...
Added by yanivkalfa on Thu, 27 Jan 2022 07:08:22 +0200