1-844-696-6465 (US)        +91 77600 44484        help@dezyre.com

Top 5 Libraries for Data Science in Python

If you are an aspiring data scientist- always learning, exploring and playing with data then this blog post will help you get ready to begin your career in data science with Python. Python Language has a rich and healthy ecosystem with ample of libraries for data analysis, data I/O and data munging. The best way to make sure that you are all set to become a data scientist is to make yourself well-versed with the various Python libraries and tools that people use in the industry for doing data science. We asked our data science faculty to list five Python libraries for data science that they think every data scientist must know how to use. Check them out below:

Python Libraries for Data Science

Python Libraries for Data Science

1) Pandas

All of us can easily do some kind of data analysis using pen and paper on small data sets. Imagine a situation, where we have to analyze millions of petabytes of data. We would require specialized tools and techniques to analyze and derive meaningful information from huge datasets. Pandas Python is one of those libraries for data analysis, that contains high-level data structures and tools to help data scientists or data analysts manipulate data in a very simple and easy way.

Work on Hands on Projects in Big Data and Data Science

To provide a very simple and yet effective way to analyze data requires the ability to index, retrieve, split, join, restructure and various other analysis on both multi and single dimensional data. Pandas data analysis library has some unique features that provide these capabilities-

i) The Series and DataFrame Objects

These two are high performance array and table structures, for representing the heterogeneous and homogeneous data sets in Pandas Python.

ii) Restructuring of  Data Sets

Pandas python provides the flexibility for reshaping the data structures so that the data can be inserted in both rows and columns of tabular data.

iii) Labelling

To allow automatic alignment of data and indexing, pandas provides labelling on series and tabular data.

iv) Multiple Labels for a Data Item

Heterogeneous indexing of data spread across multiple axes, which helps in creating more than one label on each data item.

v) Grouping

The functionality to perform split-apply-combine on series as well on tabular data.

vi) Identify and Fix Missing Data

Using pandas, programmers can easily identify and mix missing data in both floating and non-floating pointing numbers.

vii) Powerful capabilities to load and save data from various formats such as JSON, CSV, HDF5, etc.

viii) Conversion from NumPy and Python data structures to pandas objects.

ix) Slicing and sub setting of datasets, which include merging and joining data sets with SQL- like constructs.

Although, pandas provides many statistical methods, it merely is not enough for doing data science in Python. Pandas depends upon other python libraries for data science like NumPy, SciPy, Sci-Kit Learn, Matplotlib, ggvis in the Python ecosystem to draw conclusions from large data sets. Thus, making it possible for Pandas applications to take advantage of the robust and extensive Python framework.

Installation of Pandas

There are many people who often say that “Python is amazing for doing data science but they have spent 3 days installing Python and other libraries to learn doing data science in Python.” It is not recommended to install the PyData stack manually particularly when one does not know which libraries they will actually need. If you are one among them, then Anaconda by Continuum is for you.

Anaconda, is one of the most popular Python distribution offering both paid and free components. Anaconda is very popular among the open source community because of its cross-platform support to run on Windows, Mac or Linux

The base package of Anaconda installs pandas as a part of the default installation process, which makes it easy to begin using pandas. The default installation also installs IPython Notebook server, which can be used to run the applications interactively.

Excited? Now, let’s install Anaconda and pandas, to write some cool stuff!!

How to install Anaconda?

You can download the latest Anaconda from the Continuum Analytics website https://www.continuum.io/downloads. Once you visit the website, it will automatically detect the OS and provide you with different options for downloading.

Installing Anaconda

Installing Anaconda

After downloading the installer, Windows system provides the executable as shown below-

Anaconda Installation

After executing the installer and the screen will guide you to install Anaconda, just follow the on screen commands and finish the installation process.

Anaconda Setup

 

After the installation process is completed, open the command prompt and type python, the below screen will appear on successful installation of Python -

Python for Data Science

Now since Anaconda is installed successfully, we need to check, if pandas installed is the most recent version or not. Pandas version can be verified using conda package manager from the command line as follows-

conda list pandas

Pandas Python

If the pandas version installed is not a recent one, then use the below command to update Pandas-

conda update pandas

This command will download the latest version of pandas and all its dependencies as follows-

Python for Data Analysis

Example on Using Pandas DataFrame object

We will write our first application in IPython interpreter, as it provides a very convenient way for writing Python applications.

Open the command prompt and type ipython as shown below-

Using Pandas for DataFrame Object

In [1]: import pandas as pd

In [2]: mydf = pd.DataFrame.from_items ([('column1', [1, 2, 3])])

In [3]: print (mydf)

Output for the above code:

column1

0        1

1        2

2        3

Data Analysis with Python Pandas

That’s it! It’s very easy to write pandas applications using IPython. We can also write using the web based GUI of IPython Notebook.

Learn Data Science in Python for all your Data and Analytic Needs

2) NumPy

Numerical Python code name: - NumPy, is a Python library for numerical calculations and scientific computations. NumPy provides numerous features which can be used by Python enthusiasts and programmers to work with high-performing arrays and matrices. NumPy arrays provide vectorization of mathematical operations, which gives it a performance boost over Python’s looping constructs.

pandas Series and DataFrame objects rely primarily on NumPy arrays for all the mathematical calculations like slicing elements and performing vector operations. Below are some of the features provided by NumPy-

  1. Integration with legacy languages.
  2. Mathematical Operations: It provides all the standard functions required to perform operations on large data sets in a very fast and efficient way, which otherwise have to be performed through looping constructs.
  3. ndarray: It is a fast and efficient multidimensional array which can perform vector based arithmetic operations and has powerful broadcasting capabilities.
  4. I/O Operations: It provides various tools which can be used to write/read very large data sets from disk. It also supports I/O operations on memory based file mappings.
  5. Fourier transform capabilities, Linear Algebra and Random Number Generation.

Installation of NumPy

If you have installed Anaconda as mentioned above, then NumPy will get installed automatically, as it is one of the dependency of pandas. But, in case you have downloaded it via some other tools, then you need to download NumPy separately, after installing Python. Also, you need to keep in mind that, NumPy has to be installed first and then any other add-ons can be installed.

Fundo Stuffs in NumPy

Arrays Operations

i) Creation of Arrays

Open command prompt and type ipython. Then type below commands

In [1]: import umpy as np

In [2]: myarray = np.array([7,4,3,8,9],int)

In [3]: myarray

Out[3]: array([7, 4, 3, 8, 9])

 

                        Array Operations with NumPy

ii) Slicing An Array

In [5]: myarray[:2]

Out[5]: array([7, 4])

 

                   Slicing an Array in NumPy

iii) Accessing elements from Array

In [6]: myarray[3]

Out[6]: 8

                     Accessing Elements from an Array using NumPy 

 

iv) Converting Array to a binary string

In [1]: import numpy as np

In [2]: myarray = np.array([4,5,7,8],int)

In [3]: mystring = myarray.tostring()

In [4]: mystring

Out[4]: b'\x04\x00\x00\x00\x05\x00\x00\x00\x07\x00\x00\x00\x08\x00\x00\x00'                   

Converting an Array to Binary using NumPy Python Library

Phew!!! Those were some cool commands, let’s move forward to our next Python library in the list.

Click Here to Enrol Now for Free  Introductory Data Science in Python Course

 

3) SciPy

Scientific Python code name, SciPy-It is an assortment of mathematical functions and algorithms which are built on top of Python’s extension NumPy. SciPy provides various high-level commands and classes for manipulating and visualizing data. SciPy is useful for data-processing and prototyping of systems.

Apart from this, SciPy provides other advantages for building scientific applications and many specialized sophisticated applications that are backed by powerful and fast growing Python community.

Installing SciPy

As we are using Anaconda for installing Python modules and running the commands, for SciPy also we will be using Anaconda. Being a dependency for pandas, SciPy also gets installed with the default installation of Anaconda. For Python lovers, SciPy is also available for downloading separately using this http://www.scipy.org/install.html link.

Exploring the Power of SciPy with some Examples-

Linear Algebra

i) Creating an array

In [1]: import numpy as np

In [2]: import scipy as sp

In [3]: import scipy.linalg as spalg

In [4]: myarray1 = np.array([[1,2],[3,4]])

In [6]: myarray1

Out[6]:

array([[1, 2],

[3, 4]])

 

Creating an Array using SciPy

ii) Calculating the Inverse of an Array

In [7]: spalg.inv(myarray1)

Out[7]:

array ([[-2. , 1. ],

[ 1.5, -0.5]])

Inverse of an Array using SciPy

iii) Matrix Multiplication

In [8]: myarray2 = np.array([[5,6]])

In [11]: myarray1.dot(myarray2.T)

Out[11]:

array([[17],

[39]])

SciPy Python Library for Data Science

Awesome!!! We have just have completed three most popular libraries of Python for data science and it’s time to go ahead with the next one.

4) Matplotlib

We all have heard this quote “Necessity is the mother of all invention”.  The same holds true for matplotlib. This open source project was developed to handle different types of data generated from multiple sources in the epilepsy. matplotlib is a 2D graphical Python library. However, it also supports 3D graphics, but this is very limited. With increasing demand for Python in many folds in the recent years, growth of matplotlib has given tough competition to giants like MATLAB and Mathematica.

Installation of Matplotlib

You might be very happy to hear this that, when you installed Anaconda, matplotlib has also got installed under default installation. So, you do not need to do any extra installation. For knowledge seekers who want to build through the source code of Matplotlib, visit http://matplotlib.org/users/installing.html.

Time to do Some Plotting using Matplotlib

Demo for creating date plots, loading a default Yahoo csv file which comes with default installation.

In [1]: import datetime as dt

In [2]: import numpy as np

In [3]: import matplotlib.pyplot as matpy

In [4]: import matplotlib.dates as matdt

In [5]: import matplotlib.cbook as cbook

In [6]: yrs = matdt.YearLocator()

In [7]: mnt = matdt.MonthLocator()

In [8]: yrsFmt = matdt.DateFormatter('%Y')

In [9]: dataFile = cbook.get_sample_data('goog.npy')

In [10]: try:

 r = np.load(dataFile,encoding='bytes').view(np.recarray)

 except TypeError:

 r = np.load(dataFile).view(np.recarray)

In [13]: fig,ax = matpy.subplots()

In [14]: ax.plot(r.date,r.adj_close)

Out[14]: []

In [15]: ax.xaxis.set_major_locator(yrs)

In [16]: ax.xaxis.set_major_formatter(yrsFmt)

In [17]: ax.xaxis.set_minor_locator(mnt)

In [19]: mindate = dt.date(r.date.min().year,1,1)

In [20]: maxdate = dt.date(r.date.max().year+1,1,1)

In [21]: ax.set_xlim(mindate,maxdate)

Out[21]: (731581.0, 733408.0)

In [22]: def price(x):

 return '$%1.2f' % x

In [23]: ax.format_xdata = matdt.DateFormatter('%Y-%m-%d')

In [25]: ax.format_ydata = price

In [26]: ax.grid(True)

In [27]: fig.autofmt_xdate()

In [28]: matpy.show()

 

 


Plotting using Matplotlib in Python

 

Matplotlib Data Science Library for Plotting in Python

 

Matplotlib-Data Science Libraries for Python

Data Science in Python using Matplotlib

Once the last command matpy.show () is executed, a pop-up window will appear with the result as shown below-

Plot using Matplotlib

With some basic coding and commands you are able to create a visual graph based on the data, imagine the brilliance and power of matplotlib.

5) Sci-Kit Learn

For all the machine learning practitioners who want to bring machine learning into the production systems, Sci-Kit Learn is the savior, Sci-Kit Learn has several supervised and unsupervised machine learning algorithms which have a level of robustness and support required for use in production applications. As this library provides various learning algorithms, it has been named as Sci-Kit Learn. Sci-Kit Learn focuses on code quality, good documentation, ease of use and performance.Sci-Kit Learn has a steep learning curve.

Installation of Sci-Kit Learn

Sci-Kit Learn is built upon SciPy and thus to use Sci-Kit Learn it is necessary to install various other Python libraries – Pandas, NumPy, SciPy, SymPy and IPython (the enhanced interactive console).However, on installing Anaconda-Sci-Kit Learn is also installed by default.

Learn Data Science in Python to build predictive models

At the beginning of this article you might have heard only about the popular libraries in python for data science but now you can do some basic coding and make wonders using Python libraries with your datasets. Python ecosystem is a huge ocean with so many libraries to be unleashed for data scientists. These were just few of them. Subscribe to our blog for more updates on exploring other Python libraries.

CLICK HERE to get the 2016 data scientist salary report delivered to your inbox!

 

PREVIOUS

NEXT

Work on hands on projects on Big Data and Hadoop with Industry Professionals