How to Send automated email with the analysis results using python?
BIG DATA RECIPES DATA CLEANING PYTHON DATA MUNGING MACHINE LEARNING RECIPES PANDAS CHEATSHEET     ALL TAGS

How to Send automated email with the analysis results using python?

How to Send automated email with the analysis results using python?

In this Big Data recipe, we will show how to Send automated email with the analysis results using python

0

Recipe Objective

In big data scenarios , Email automation is the most important phase after any big data pipeline to send logs, error files, result sets to the business stakeholders for them to analyse the data . It enables you to enhance your email functionality without manually sending out individual emails whenever a task in a big data pipeline runs or fails . Using automated email in your email campaign helps increase engagement and provide a more tailored email experience for your recipients. For example , one can send emails with log files, result sets , validation data to other team members using email functionality in python.

System requirements

  • Install the python module as follows :
  • pip install pyzmail36
  • pip install smtplib
  • pip install pandas
  • pip install imapclient
  • The below codes can be run in Jupyter notebook , or any python console

Step 1: Connecting to the Gmail and logging in

  • Before using Python to access your email, we will need to change our google account setting to allow access to less secure apps.
  • Otherwise, Google will not be able to give access to your code to sign in to the gmail account. Now, for security reasons, we recommend changing this setting back to its default state once you're done running the code.
  • Turn Allow less secure apps to ON. Be aware that this makes it easier for others to gain access to your account. Under "Security" catalogue, find "Less secure app access" and turn it on
  • Python provides smtplib module, which defines an SMTP client session object that can be used to send mail to any Internet machine with an SMTP or ESMTP listener daemon.
  • SMTP stands for Simple Mail Transfer Protocol.
  • The smtplib module is useful for communicating with mail servers to send mail.

Step 2: Sending a Plain-text Email through the Code.

import smtplib, ssl port = 587 # For starttls smtp_server = "smtp.gmail.com" sender_email = "xxxxxxxxxx@gmail.com" receiver_email = "xxxxxxxxxcv@gmail.com" password = input("Type your password and press enter:") message = """\ Subject: Hi there Im sending an email through python code.""" context = ssl.create_default_context() with smtplib.SMTP(smtp_server, port) as server: server.ehlo() server.starttls(context=context) server.ehlo() server.login(sender_email, password) server.sendmail(sender_email, receiver_email, message)

Output of the above code:

Step 3 : Sending an Email with attachment through the Code

  • Python's email package contains many classes and functions for composing and parsing email messages, this section only covers a small subset useful for sending emails.
  • There are quite a few ways to attach files to an email, but this is one of the simpler ways. This time we will use an encoder and MIMEMultipart from the email module and os.path to simply get the filename from the provided path.
  • This method supports text files, images, videos, audio and pdfs.
import smtplib from email.mime.text import MIMEText from email.mime.multipart import MIMEMultipart from email.mime.base import MIMEBase from email import encoders import os.path email = 'your@gmail.com' password = '***********' send_to_email = 'others@gmail.com' subject = 'Sending Email with an attachment' message = 'Please find the attachment to email, thanks' file_location = 'C:\\Users\\DELL E7440\\Desktop\\pyspark - Copy.txt' msg = MIMEMultipart() msg['From'] = email msg['To'] = send_to_email msg['Subject'] = subject msg.attach(MIMEText(message, 'plain')) # Setup the attachment filename = os.path.basename(file_location) attachment = open(file_location, "rb") part = MIMEBase('application', 'octet-stream') part.set_payload(attachment.read()) encoders.encode_base64(part) part.add_header('Content-Disposition', "attachment; filename= %s" % filename) # Attach the attachment to the MIMEMultipart object msg.attach(part) server = smtplib.SMTP('smtp.gmail.com', 587) server.starttls() server.login(email, password) text = msg.as_string() server.sendmail(email, send_to_email, text) server.quit()

Output of the above code:

Pyzmail Library :

  • pyzmail is a high level mail library.
  • It provides functions and classes that help to parse, compose and send emails. pyzmail hide the difficulties of managing the MIME structure and of the encoding/decoding for internationalized emails

To install :

pip install pyzmail36

Connecting to imap_server

import imapclient import smtplib import datetime import pyzmail imap_server = 'imap.gmail.com' smtp_server = 'smtp.gmail.com' username = '******@gmail.com' password = '******' imapobj = imapclient.IMAPClient(imap_server, ssl=True) imapobj.login(username, password) smtpobj = smtplib.SMTP(smtp_server, 587) smtpobj.ehlo() smtpobj.starttls() smtpobj.login(username, password)

Searching and retrieving emails

import imaplib import pprint pprint.pprint(imapobj.list_folders()) imaplib._MAXLINE = 10000000 imapobj.select_folder('Inbox', readonly=True) UIDs = imapobj.search(['SINCE', '01-Aug-2020', 'BEFORE', '01-Sep-2020'])

Step 4 : Write inbox emails details to csv using pyzmail library.

import pyzmail import csv with open('emaildata.csv', 'a', newline='', encoding='utf-8') as mycsvFile: for i in range(len(UIDs)): raw_message = imapobj.fetch(UIDs[i], ['BODY[]']) message = pyzmail.PyzMessage.factory(raw_message[UIDs[i]][b'BODY[]']) email_sub = message.get_subject() email_from = message.get_addresses('from') email_to = message.get_addresses('to') mail_date = message.get_decoded_header('date') mail_date = mail_date.split("<")[0].replace('"', '') for eid in UIDs: mycsvFileWriter = csv.writer(mycsvFile) data = [eid, mail_date, email_from[0][0], email_from[0][1],email_to[0][1], email_sub] mycsvFileWriter.writerow(data) print('done')

Output of the Above code:

It will create a csv file with email Id, from emails , to- emails and subject data and it will write the csv file to the current location where python is being run.

Read csv file which is written in the above step and using pandas print the first 5 lines

import pandas as pd df = pd.read_csv("mycsvfile.csv") df.head()

Output of the above code :

Adding the column names to the dataframe which we created from csv file.

df.columns =['eid', 'mail_date','sender_name','email_from', 'email_to', 'subject'] df.head()

Output of the above code: After adding the columns printing the first 5 lines

Taking Unique emails of senders and creating another dataframe from the csv file.

Sender = df['email_from'].unique().tolist() Sender_emaildata = pd.DataFrame({ 'Senders': Sender })

After creating unique emails dataframe write and encode dataframe to csv file to the e-mail using pandas

import pandas as pd Sender_emaildata.to_csv('unique_emails.csv', encoding='utf-8')

Output of the above code :

We can send the email by attaching the csv file which is generated using the above step

## Sending an Email with attachment through the Code. import smtplib from email.mime.text import MIMEText from email.mime.multipart import MIMEMultipart from email.mime.base import MIMEBase from email import encoders import os.path email = '******@gmail.com' password = '**********' send_to_email = '*******@gmail.com' subject = 'Sending Email with an attachment' message = 'Please find the attachment to email, thanks' file_location = 'unique_emails.csv' msg = MIMEMultipart() msg['From'] = email msg['To'] = send_to_email msg['Subject'] = subject msg.attach(MIMEText(message, 'plain')) # Setup the attachment filename = os.path.basename(file_location) attachment = open(file_location, "rb") part = MIMEBase('application', 'octet-stream') part.set_payload(attachment.read()) encoders.encode_base64(part) part.add_header('Content-Disposition', "attachment; filename= %s" % filename) # Attach the attachment to the MIMEMultipart object msg.attach(part) server = smtplib.SMTP('smtp.gmail.com', 587) server.starttls() server.login(email, password) text = msg.as_string() server.sendmail(email, send_to_email, text) server.quit()

Output of the above code :

Relevant Projects

PySpark Tutorial - Learn to use Apache Spark with Python
PySpark Project-Get a handle on using Python with Spark through this hands-on data processing spark python tutorial.

Data Warehouse Design for E-commerce Environments
In this hive project, you will design a data warehouse for e-commerce environments.

Yelp Data Processing using Spark and Hive Part 2
In this spark project, we will continue building the data warehouse from the previous project Yelp Data Processing Using Spark And Hive Part 1 and will do further data processing to develop diverse data products.

Analyse Yelp Dataset with Spark & Parquet Format on Azure Databricks
In this Databricks Azure project, you will use Spark & Parquet file formats to analyse the Yelp reviews dataset. As part of this you will deploy Azure data factory, data pipelines and visualise the analysis.

Explore features of Spark SQL in practice on Spark 2.0
The goal of this spark project for students is to explore the features of Spark SQL in practice on the latest version of Spark i.e. Spark 2.0.

Online Hadoop Projects -Solving small file problem in Hadoop
In this hadoop project, we are going to be continuing the series on data engineering by discussing and implementing various ways to solve the hadoop small file problem.

Real-Time Log Processing using Spark Streaming Architecture
In this Spark project, we are going to bring processing to the speed layer of the lambda architecture which opens up capabilities to monitor application real time performance, measure real time comfort with applications and real time alert in case of security

Hadoop Project for Beginners-SQL Analytics with Hive
In this hadoop project, learn about the features in Hive that allow us to perform analytical queries over large datasets.

Create A Data Pipeline Based On Messaging Using PySpark And Hive - Covid-19 Analysis
In this PySpark project, you will simulate a complex real-world data pipeline based on messaging. This project is deployed using the following tech stack - NiFi, PySpark, Hive, HDFS, Kafka, Airflow, Tableau and AWS QuickSight.

Build a big data pipeline with AWS Quicksight, Druid, and Hive
Use the dataset on aviation for analytics to simulate a complex real-world big data pipeline based on messaging with AWS Quicksight, Druid, NiFi, Kafka, and Hive.