Introduction

When it comes to Python's libraries, you may first think of the following:

• NumPy: for matrix computing, machine learning;
• pandas: NumPy-based data processing for standard Excel/SQL tables, or Tidyverse for R;
• SciPy: Scientific calculation, standard MATLAB;
• statsmodels: Used for metrological modeling, AER for standard R, or Stata;
• skilearn: for machine learning modeling;
• PyTorch/TensorFlow/Keras: for in-depth learning modeling;
• matplotlib/plotly: for data visualization, ggplot2 for standard R;
• seaborn/Bokeh: Advanced/interactive visualization;
• turtle: based on cursors instead of data drawing, more comprehensive functionality;
• scrapy/requests/beautiful soup: web crawler;
• NLTK: Natural Language Processing
• ...

• 

In addition to this, Python has a number of feature-rich libraries, such as the following:

• emoji
• geopy
• inspect

Of course, in addition to some interesting modules, there are some unique features of Python in this article, including:

• list comprehension
• Anonymous function, lambda function
• map mapping function

1 Emotional Pack Library: emoji

The emoji module helps Python output emoticon packages/symbols.

# -*- coding: utf-8 -*-
"""
Created on Fri Jun 18 01:52:59 2021
@Software: Spyder
@author: Blind Zone Walker King
"""

from emoji import emojize

emojize(":thumbs_up:")
Out[19]: ' '

emojize("Python is fun :red_heart:")
Out[20]: 'Python is fun ❤'

2 Geographic data processing module: geopy

Official website: Welcome to Geopy Documentation!- GeoPy 2.0.0 Documentation

The geopy module provides a series of API s for geocoding services that make address acquisition and geographic information processing such as longitude, dimension, and elevation easy. On this basis, we can calculate the distance between two locations.

First, see how to define a location from dimensions, longitude, and elevation:

import geopy

p1 = geopy.point.Point(41.5, -81.0, 12.3)

str(p1)  ## View structure
Out[31]: '41 30m 0s N, 81 0m 0s W, 12.3km'

Next, we try to calculate the distance between the two places:

from geopy import distance
newport_ri = (41.49008, -71.312796)  ##Define Location 1
cleveland_oh = (41.499498, -81.695391)  ##Define Location 2
distance.distance(newport_ri, cleveland_oh).kilometers ##In kilometers

Out[21]: 866.4554329098687

3 Python's own inspect

The inspect module helps us understand the source code behind Python and the class structure.

import inspect
inspect.getsource(inspect.getsource)
Out[35]: 'def getsource(object):\n    """Return the text of the source code for an object.\n\n    The argument may be a module, class, method, function, traceback, frame,\n    or code object.  The source code is returned as a single string.  An\n    OSError is raised if the source code cannot be retrieved."""\n    lines, lnum = getsourcelines(object)\n    return \'\'.join(lines)\n'

4 News web crawl package: newspaper3k

This package combines the functionality of a crawler, NLP, to help us retrieve news articles from a large number of international publications, along with the corresponding metadata. From the news article data, we can retrieve specific names, keywords, images, and so on.

## Retrieving individual news based on rticle method

from newspaper import Article

url = "https://edition.cnn.com/2021/06/18/china/billion-vaccine-shots-mic-intl-hnk/index.html"

news = Article(url, language='en')

news.download()        ##Download Web Page

news.parse()           ##Parse the page

news.title      ## Title
Out[98]: "China's about to administer its billionth coronavirus shot. Yes, you read that right"

news.text      ## News Body
Out[99]: '(CNN) Editor\'s note: CNN is launching the Meanwhile in China newsletter on June 21, a ##Very long, omitted.

news.authors     ## author
Out[100]: ['Nectar Gan', 'Laura He']

news.keywords    ## Keyword, result is empty
Out[101]: []

news.summary     ## abstract
Out[102]: ''

news.top_image ## Map URL
Out[103]: 'https://cdn.cnn.com/cnnnext/dam/assets/210618002412-china-coronavirus-vaccine-0603-super-tease.jpg'

news.publish_date ## Press release date
Out[104]: datetime.datetime(2021, 6, 18, 0, 0)

5 CPU and Memory Analysis Pack Scalene

Official website: scalene

scalene: An excellent dedicated module for viewing CPU, GPU, and memory performance.

However, during the installation, I did not install it in half a day, so here is the description of the official website, you can see.

6 Web Visualization Package: pyecharts

The difference between this package and matplotlib is how easy it is to make graphics in html format. pyecharts can also further integrate with popular Web frameworks such as Flask and Django, and support geographic data visualization based on Baidu Maps.

from pyecharts.charts import Bar

bargraph = Bar()
bargraph.add_xaxis(["Economics", "Finance", "Financial Engineering", "Stata", "Python"])
bargraph.add_yaxis("Classmate Wang, a pedestrian in blind area", [91, 92, 93, 94, 95])

render
bargraph.render()

7 Beautiful and simple progress bar: tqdm

Official website: tqdm/tqdm

tqdm, from Arabic "taqaddum", = progress, meaning process/progress.

Through tqdm, you can see the progress of the algorithm in the specific execution process. Especially among algorithms that take a long time to run (such as bubble sort 1 million integers), we can see in real time the proportion of the algorithm completed.

Demo code:

from tqdm import tnrange, tqdm_notebook
from time import sleep

for i in tqdm_notebook(tnrange(4), desc = "Loop on Layer 1"):
    for j in tqdm_notebook(tnrange(100), desc = "Loop on Layer 2", leave = False):
        sleep(0.02)

8 List Derivation List Comprehension

The list derivation is a bit like the anonymous function described below, which you can learn by comparison. They can greatly improve the efficiency of Python.

a = [1, 2, 3, 4, 5]

b = [x**3 for x in a]  ##List Derivation

b
Out[49]: [1, 8, 27, 64, 125]

c = [x**3 for x in a if x%2 == 0]

c
Out[51]: [8, 64]

9 lambda functions and map mappings

The lambda function, also known as the anonymous function, is a function different from the ordinary function. With the map() function, new elements generated by the lambda function can be converted to objects such as lists or tuples.

a = [1, 2, 3, 4, 5]

b = map(lambda a:a**3, a)

list(b) ##Convert to List
Out[46]: [1, 8, 27, 64, 125]