How to use Adam optimizer using TF learn
This recipe helps you use Adam optimizer using TF learn
Recipe Objective
This recipe explains how to use Adam optimizer using TFLayer.
Explore the BERT Variants - ALBERT vs DistilBERT
Adam
Its syntax is tflearn.optimizers.Adam (learning_rate=0.001, beta1=0.9, beta2=0.999, epsilon=1e-08, use_locking=False, name='Adam')
where its arguments learning_rate which is learning rate, use_locking which if true uses locks for update operation, name which is the optional name prefix for the operations, beta1 and beta2 which are the exponential decay rates for the 1st moment estimates and 2nd moment estimates.
class Adam(Optimizer):
def __init__(self, lr=0.001, b1=0.9, b2=0.999, e=1e-8, ul=False, N="Adam"):
super(Adam, self).__init__(lr, ul, N)
self.e = e
self.b1 = b1
self.b2 = b2
def build(self, st=None):
self.Tensor = tf.train.AdamOptimizer( lr=self.lr, b1=self.b1, b2=self.b2, e=self.e, ul=self.ul, N=self.N)
self.Built = True
adam = Adam
What Users are saying..
Abhinav Agarwal
I come from Northwestern University, which is ranked 9th in the US. Although the high-quality academics at school taught me all the basics I needed, obtaining practical experience was a challenge.... Read More