HiT models are used to encode entities (presented as texts) and predict their hierarhical relationships in hyperbolic space.
Get Started
Install hierarchy_transformers (check our repository) through pip or GitHub.
Use the code below to get started with the model.
python
from hierarchy_transformers import HierarchyTransformer
# load the model
model = HierarchyTransformer.from_pretrained('Hierarchy-Transformers/HiT-MiniLM-L12-WordNetNoun')
# entity names to be encoded.
entity_names = ["computer", "personal computer", "fruit", "berry"]
# get the entity embeddings
entity_embeddings = model.encode(entity_names)
Default Probing for Subsumption Prediction
Use the entity embeddings to predict the subsumption relationships between them.
python
# suppose we want to compare "personal computer" and "computer", "berry" and "fruit"
child_entity_embeddings = model.encode(["personal computer", "berry"], convert_to_tensor=True)
parent_entity_embeddings = model.encode(["computer", "fruit"], convert_to_tensor=True)
# compute the hyperbolic distances and norms of entity embeddings
dists = model.manifold.dist(child_entity_embeddings, parent_entity_embeddings)
child_norms = model.manifold.dist0(child_entity_embeddings)
parent_norms = model.manifold.dist0(parent_entity_embeddings)
# use the empirical function for subsumption prediction proposed in the paper
# `centri_score_weight` and the overall threshold are determined on the validation set
subsumption_scores = - (dists + centri_score_weight * (parent_norms - child_norms))
Train Your Own Models
Use the example scripts in our repository to reproduce existing models and train/evaluate your own models.
Yuan He, Zhangdie Yuan, Jiaoyan Chen, Ian Horrocks.Language Models as Hierarchy Encoders. Advances in Neural Information Processing Systems 37 (NeurIPS 2024).
text
@article{he2024language,
title={Language models as hierarchy encoders},
author={He, Yuan and Yuan, Moy and Chen, Jiaoyan and Horrocks, Ian},
journal={Advances in Neural Information Processing Systems},
volume={37},
pages={14690--14711},
year={2024}
}
Model Card Contact
For any queries or feedback, please contact Yuan He (yuan.he(at)cs.ox.ac.uk).
â ī¸ Incomplete Data
Some information about this model is not available.
Use with Caution - Verify details from the original source before relying on this data.