Teach you how to use Python web crawler to send mail regularly (with source code)

Hello, I'm meow in the IT industry.

preface

A few days ago, a big man shared a code to grab the reading directory and send emails regularly in the group. I feel it's quite good. I'll share it here for you to learn.

1, Train of thought

The idea is not difficult. Construct a crawler task, grab the directory on a book website, then return the obtained content, and then use Python to realize the function of sending mail. The rest is the construction of timed tasks. Let's take a look at the specific implementation process.

2, Concrete implementation

The code of the boss is directly lost here, as shown below:

# -*- coding: utf-8 -*-
import requests, bs4
import smtplib
import schedule
import time
from bs4 import BeautifulSoup
from email.mime.text import MIMEText
from email.header import Header


# account = '{0}'.format('sender qq mailbox ')
# password = '{0}'.format('qq mailbox authorization code ')
# receiver = '{0}'.format('recipient 163 mailbox or qq mailbox ')
account = '{0}'.format('2352180977@qq.com')
password = '{0}'.format('awmowqginzdijg')
receiver = '{0}'.format('2352180977@qq.com')


# The crawler task is to obtain the title and author of the book on the sobooks website, and the page can be modified according to your own needs
def recipe_spider():
    list_all = ''
    num = 0
    for a in range(1, 3):
        n = '{0}{1}'.format('https://sobooks.cc/page/', a)
        res = requests.get(n)
        res.encoding = res.apparent_encoding
        bs = BeautifulSoup(res.text, 'html.parser')
# print(bs)
    
        books = bs.find_all('h3')
        authors = bs.find_all('p')
        
        for i in range(len(books)):
            num = num+1
            book = books[i].text.strip()
            author = authors[i+1].text.strip()
        #list_books.append([book,author])
#  list_books.append(list_book)

            n = '''
 title%s: %s,author: %s
            ''' % (num, book, author)
            list_all = list_all + n
    return list_all


# Send the obtained content to e-mail
def send_email(list_all):
    global account, password, receiver
    mailhost = 'smtp.qq.com'
    qqmail = smtplib.SMTP()
    qqmail.connect(mailhost, 25)
    qqmail.login(account, password)
    content = 'Honey, today's book list' + list_all
    print(content)
    message = MIMEText(content, 'plain', 'utf-8')
    subject = 'What are you watching today'
    message['Subject'] = Header(subject, 'utf-8')
    try:
        qqmail.sendmail(account, receiver, message.as_string())
        print('Mail sent successfully')
    except:
        print('Mail sending failed')
    qqmail.quit()


def job():
    print('Start a task')
    list_all = recipe_spider()
    send_email(list_all)
    print('Mission accomplished')


if __name__ == '__main__':
    # Scheduled task, where 0.05 means interval, with minutes as the interval, and the time is an integer by default.
    schedule.every(0.05).minutes.do(job)
    while True:
        schedule.run_pending()
        time.sleep(1)

After the program runs, the mail can be sent automatically.

Wait a moment, and the email will be sent to you automatically. The following figure is the email notification.

The following figure shows the specific contents of the email: 

For the above code, you only need to change three places, one is the sender's qq email, the other is the authorization code of qq email, and the third is the recipient's email, as shown in the red box below.

3, Frequently asked questions

Small and medium-sized partners should often encounter this problem during operation, as shown in the figure below.

 

This is probably because your mailbox is not filled in correctly or the authorization code is wrong. Check whether the mailbox suffix is added or whether the authorization code is copied completely.

Some small partners don't know how to obtain the qq email authorization code. Under the guidance here, first you have to open your qq email, and then click Set -- > account, as shown in the figure below:

Pull to the bottom, as shown in the figure below:

Click generate authorization code, and the following interface will pop up:

The mobile phone sends the keyword "configure email client". Remember, it is "email", not "email". Xiaobian failed because he sent the wrong word before. On the importance of care! After sending successfully, the following figure will pop up:

The letter part of the bar box on the left of the figure above is the authorization code of qq mailbox. Copy it into the code and paste it.

4, Summary

This article mainly introduces you to the small project of using Python web crawler to automatically and regularly send mail. The main idea of implementation is to construct a crawler task, grab the directory on a book website, then return the obtained content, then use Python to realize the function of e-mail sending, and build a timed task. Finally, we give you examples of how to deal with common problems.

Guys, practice it quickly! If you encounter any problems in the learning process, please pay attention to me~

Keywords: Python Multithreading crawler Flask request

Added by doublehops on Thu, 23 Dec 2021 22:48:12 +0200