How to create Lookup parameters using dynet

This recipe helps you create Lookup parameters using dynet

Recipe Objective - How to create Lookup parameters using dynet?

LookupParameters are used to embed a set of discrete objects. For Example - Word Embedding. LookupParameters is used to represents a table of parameters.

In "ParameterCollection()" module we have "add_lookup_parameters" function which used to create a lookup parameters.

Creating LookupParameters:-

import dynet as dy
import numpy as np

# creating a parameter collection and adding the parameters.
pc = dy.ParameterCollection()

VOCAB_ROW = 100 # VOCAB_ROW is used as a ROW
DIMN = 10 # DIMN is used as a dimensions
lp = pc.add_lookup_parameters((VOCAB_ROW, DIMN))

Creating expressions from lookup parameters:-

exp7 = dy.lookup(lp, 7) # creating an Expression from row 7.
exp7 = lp[7]
exp7c = dy.lookup(lp, 7, update=False) # as before, but don't update when optimizing.

exp67 = dy.lookup_batch(lp, [6, 7]) # creating a batched Expression from rows 6 and 7.
exp67 = lp.batch([6, 7])
print('exp67 dim:', exp67.dim())

exp1_19 = dy.lookup_batch(lp, range(1, 20)) # creating a batched Expression from rows 1 to 19
exp1_19 = lp.batch(range(1, 20))
print('exp1_19 dim:', exp1_19.dim())

exp5.set(10) # now the exp5 expression contains row 10
print('exp5 dim after applying set method', exp5.dim())
print(exp5.value())

# We can check if it is actually containing row 15
exp10 = lp[15]
print(exp5.value() == exp10.value())

Initialize LookupParameters:-

scale, mean, stddev = 1, 0, 1

# Creates 2x4 matrix filled with 0 (or any other float)
lp1 = pc.add_lookup_parameters((2,4), init=0)

# Creates 2x4 matrix initialized with U([-scale, scale])
lp2 = pc.add_lookup_parameters((2,4), init='uniform', scale=scale)

# Creates 2x4 matrix initialized with N(mean, stddev)
lp3 = pc.add_lookup_parameters((2,4), init='normal', mean=mean, std=stddev)

# Creates 4x4 identity matrix
lp4 = pc.add_lookup_parameters((4,4), init='identity')

# Creates 2x4 matrix with glorot init
lp5 = pc.add_lookup_parameters((2,4), init='glorot')

lp6 = pc.add_parameters((2,4)) # By default, the init = 'glorot'

# Creates 2x4 matrix with he init
lp7 = pc.add_lookup_parameters((2,4), init='he')

# Creates 2x4 matrix from a numpy array (size is inferred)
lp8 = pc.add_lookup_parameters((2,4), np.ones((2,4)))

What Users are saying..

profile image

Ray han

Tech Leader | Stanford / Yale University
linkedin profile url

I think that they are fantastic. I attended Yale and Stanford and have worked at Honeywell,Oracle, and Arthur Andersen(Accenture) in the US. I have taken Big Data and Hadoop,NoSQL, Spark, Hadoop... Read More

Relevant Projects

Credit Card Fraud Detection as a Classification Problem
In this data science project, we will predict the credit card fraud in the transactional dataset using some of the predictive models.

Build a Music Recommendation Algorithm using KKBox's Dataset
Music Recommendation Project using Machine Learning - Use the KKBox dataset to predict the chances of a user listening to a song again after their very first noticeable listening event.

MLOps using Azure Devops to Deploy a Classification Model
In this MLOps Azure project, you will learn how to deploy a classification machine learning model to predict the customer's license status on Azure through scalable CI/CD ML pipelines.

Time Series Forecasting with LSTM Neural Network Python
Deep Learning Project- Learn to apply deep learning paradigm to forecast univariate time series data.

OpenCV Project for Beginners to Learn Computer Vision Basics
In this OpenCV project, you will learn computer vision basics and the fundamentals of OpenCV library using Python.

Time Series Analysis with Facebook Prophet Python and Cesium
Time Series Analysis Project - Use the Facebook Prophet and Cesium Open Source Library for Time Series Forecasting in Python

Tensorflow Transfer Learning Model for Image Classification
Image Classification Project - Build an Image Classification Model on a Dataset of T-Shirt Images for Binary Classification

Learn to Build a Polynomial Regression Model from Scratch
In this Machine Learning Regression project, you will learn to build a polynomial regression model to predict points scored by the sports team.

Build a Speech-Text Transcriptor with Nvidia Quartznet Model
In this Deep Learning Project, you will leverage transfer learning from Nvidia QuartzNet pre-trained models to develop a speech-to-text transcriptor.

Build a Similar Images Finder with Python, Keras, and Tensorflow
Build your own image similarity application using Python to search and find images of products that are similar to any given product. You will implement the K-Nearest Neighbor algorithm to find products with maximum similarity.