Impala invalidate metadata



0

How to check how many objects are invalid in impala and require invalidte metadata or if any underlying table changed in structure how will I get how many views are affected and invalidated?


1 Answer(s)


0

Hi Chetan,

I am not sure whether is there a way to filter the invalid objects in impala.

INVALIDATE METADATA is required when the following changes are made outside of Impala, in Hive and other Hive client, such as SparkSQL:
  • Metadata of existing tables changes.
  • New tables are added, and Impala will use the tables.
  • The SERVER or DATABASE level Sentry privileges are changed.
  • Block metadata changes, but the files remain the same (HDFS rebalance).
  • UDF jars change.
  • Some tables are no longer queried, and you want to remove their metadata from the catalog and coordinator caches to reduce memory requirements.

The INVALIDATE METADATA statement marks the metadata for one or all tables as stale. The next time the Impala service performs a query against a table whose metadata is invalidated, Impala reloads the associated metadata before the query proceeds. As this is a very expensive operation compared to the incremental metadata update done by the REFRESH statement, when possible, prefer REFRESH rather than INVALIDATE METADATA.

Hope this helps.

Please refer the following link for more details: https://www.cloudera.com/documentation/enterprise/5-14-x/topics/impala_invalidate_metadata.html