Regions for Set Inclusion

Published April 3, 2026 · Revised July 24, 2026

The claim that a dog is an animal is a statement about sets: every dog is an animal, so the set of dogs is contained in the set of animals. A model may learn a point-based score correlated with that relation, but ordinary point geometry does not make inclusion literal. Region representations can make inclusion literal.

Five questions recur, and a result about one does not settle the others:

The article first builds containment and intersection as representation operations, then addresses the optimization failure of hard boxes. Query2Box and EL⁺⁺ are later, parallel applications: one composes answer-set queries, while the other scores normalized ontology axioms.

# Background

# Knowledge graph embeddings

A knowledge graph is a collection of triples $(h, r, t)$: head entity, relation, tail entity. (“Berlin”, capitalOf, “Germany”) or (“Dog”, isA, “Animal”). These graphs are large but sparse: when Freebase was active, 71% of people had no recorded birthplace and 75% had no recorded nationality (West et al., 22). One frequently studied task is link prediction: given (Berlin, capitalOf, ?) or (?, isA, Animal), rank the missing entity.

Many approaches since 2013 embed entities as vectors and score triples geometrically: either by treating relations as transformations (TransE, RotatE) or as bilinear forms like $h^\top M_r t$ where $M_r$ is a learnable matrix per relation (DistMult, ComplEx).The point-embedding models discussed here are implemented in tranz, named after the Trans* family (TransE, TransR, TransH, TransD). They can learn useful scores for hierarchical triples, but their native geometric operation is not set containment. The region embeddings that follow are in subsume, named for the subsumption relation ($\sqsubseteq$).

TransE 2, short for “translating embeddings,” models each relation as a translation: score a triple $(h, r, t)$ by $\|h + r - t\|$.Lower scores indicate a closer fit. Implementations commonly use either an L1 or L2 norm. If the triple is true, the head plus the relation vector should land near the tail. Its small parameterization and inexpensive score made it a widely used baseline.

Under the exact translation equations, TransE degenerates on some one-to-many and symmetric relations. For (Britain, hasCity, ?), each correct tail would satisfy $h+r=t$, forcing London, Manchester, and Edinburgh to the same point. For a symmetric pair, $h+r=t$ and $t+r=h$ imply $r=0$. Margin-based training relaxes these equalities, but it does not make either relation pattern native to the translation geometry.

RotatE 7, short for “rotation embedding,” fixes the symmetry problem by working in complex space: each relation is an element-wise rotation $t = h \circ r$ where $|r_i| = 1$. A symmetric relation requires $r_i^2=1$, so each component can have phase $0$ or $\pi$. This handles symmetry, antisymmetry, inversion, and composition.

Neither TransE nor RotatE makes containment a native geometric operation.A learned distance score can rank hierarchical triples successfully. What it does not supply is literal set inclusion: “close to Animal” and “inside Animal” are different geometric statements. Both embed each entity as a point. A point has no volume, interior, or boundary of its own, so one entity’s representation cannot be inside another’s.

# What containment requires

Consider the subsumption hierarchy: Dog $\sqsubseteq$ Animal $\sqsubseteq$ LivingThing, where $\sqsubseteq$ (read “is subsumed by”) means every instance of the left concept is also an instance of the right. In set-theoretic terms, $\text{ext}(\text{Dog}) \subseteq \text{ext}(\text{Animal}) \subseteq \text{ext}(\text{LivingThing})\text{,}$ where $\text{ext}(C)$ is the set of individuals that fall under concept $C$.

An embedding that captures this needs three properties:

1. Volume. More general concepts (Animal) should have larger representations than specific ones (Dog). “Animal” covers more ground than “Dog.”

2. Containment. The representation of Dog should be geometrically inside the representation of Animal. Not just “close to,” but inside.

3. Intersection. The concepts “Animal” and “Pet” overlap (some animals are pets, some aren’t). Their representations should intersect, and the intersection should itself be a valid representation, of the concept $\text{Animal} \sqcap \text{Pet}$, whether or not that conjunction has a name in the ontology.

Points do not supply these region operations. Order embeddings instead encode the partial order directly.

# Order embeddings

Vendrov et al. 3 embedded the partial order directly into the geometry. Their approach mapped concepts into the non-negative orthant $\mathbb{R}^d_+$ (the region where all coordinates are $\ge 0$, the $d$-dimensional analogue of the first quadrant) and imposed the reverse product order: $x$ is more general than $y$ if $x_i \le y_i$ for every coordinate. The origin is the top element, the most general concept.The “entity” concept, containing everything, sits at zero. Specificity grows with coordinate magnitude. Each point defines two cones: the cone extending toward the origin (smaller coordinates) contains its ancestors (more general concepts); the cone extending away from the origin (larger coordinates) contains its descendants (more specific concepts).

Reproduce this figure

order_cones.typ_diagram-style.typtypst compile order_cones.typ order_cones.png --ppi 500 --root ..

The coordinate order is transitive: if $x \le y$ and $y \le z$ coordinate-wise, then $x \le z$. The representation is still points, and the cones are unbounded. Vilnis et al. 4 proved a limitation: for any product probability measure $p(x)=\prod_{i} p_{i}(x_{i})$ over the non-negative orthant, the covariance of the indicator functions of any two cones is non-negative, $\text{Cov}(\mathbf{1}_{C_A},\mathbf{1}_{C_B})\ge0$. If “Mammal” and “Reptile” are both under “Animal,” the model cannot make them negatively correlated or disjoint under that product measure.

The forced positive covariance between sibling cones motivated boxes: bounded regions closed under intersection, with a conditional-overlap containment score.

# Box Embeddings

Reproduce this figure

box_taxonomy.typ_diagram-style.typtypst compile box_taxonomy.typ box_taxonomy.png --ppi 500 --root ..

The idea of representing words as regions rather than points goes back to Erk 1, who embedded words as convex regions in vector space to model graded entailment. Vilnis et al. 4 adapted the idea to knowledge graphs with axis-aligned hyperrectangles, boxes, in $\mathbb{R}^d$. A box $B$ is parameterized by its minimum and maximum corners:

$$B = \{x \in \mathbb{R}^d : m_i \le x_i \le M_i, \; i = 1, \ldots, d\}$$

where $m = (m_1, \ldots, m_d)$ and $M = (M_1, \ldots, M_d)$ with $m_i \le M_i$ for each coordinate.

One region family supporting all three properties is the axis-aligned box. Volume is the product of side lengths:

$$\text{Vol}(B) = \prod_{i=1}^{d} (M_i - m_i)$$

Containment is coordinate-wise: $A \subseteq B$ if and only if $m^B_i \le m^A_i$ and $M^A_i \le M^B_i$ for all $i$. The intersection of two boxes is a box (or empty):

$$(A \cap B)_i = [\max(m^A_i, m^B_i), \; \min(M^A_i, M^B_i)]$$

The intersection is non-empty when $\max(m^A_i, m^B_i) \le \min(M^A_i, M^B_i)$ for every coordinate. If any coordinate has an empty interval, the boxes are disjoint. Disjoint boxes represent mutually exclusive concepts, the thing order embeddings could not express. And when boxes partially overlap, the intersection region is itself a box representing the conjunction: the box for “Animal” intersected with the box for “Pet” gives a box for “things that are both animals and pets.”

The axis-alignment is a deliberate restriction.The intersection of two balls is generally a lens rather than a ball. ELEmbeddings (Kulmanov et al., 2019), which used n-balls for EL⁺⁺ concepts, therefore approximated conjunction ($C_1 \sqcap C_2$, NF2 in Box²EL’s numbering). Boxes instead keep intersections in the same family. Rotated boxes would be more expressive, but their intersection is not necessarily a rotated box, so they lose that closure. Axis-aligned boxes ordered by inclusion form a lattice: their meet is intersection and their join is the smallest axis-aligned bounding box containing both. That join generally contains points not in the set union, so lattice closure should not be confused with exact closure under logical disjunction. Each dimension contributes an independent “vote” on containment, the box analogue of diagonal covariance.

# Conditional overlap

The geometric intuition is that a broad concept (Animal) gets a large box and a narrow concept (Labrador Retriever) gets a smaller box inside it. When training makes the inclusion constraints hold, volume becomes a model-side proxy for generality: a subset cannot have greater volume than its superset. The converse does not hold; a large box is not, by size alone, semantically general.

One score for subsumption is the fraction of $B$ covered by $A$:

$$s(A\mid B) =\frac{\text{Vol}(A \cap B)}{\text{Vol}(B)} =P_{X\sim\text{Unif}(B)}(X\in A).$$

Concretely: draw a point uniformly from $B$ and ask whether it lands in $A$. The event $B\subseteq A$ itself is deterministic; the ratio is not the probability that one fixed set contains another. It returns 1 if $B$ is entirely inside $A$, 0 if they are disjoint, and an intermediate value for partial overlap. To express “Dog is-a Animal,” train $s(\text{Animal}\mid\text{Dog})$ toward 1.

The asymmetry is important: $s(A\mid B)\ne s(B\mid A)$ in general because the denominator changes. A small box inside a large box gives 1 in one direction and a small score in the other. This matches the semantics: every dog is an animal, but not every animal is a dog.

# A worked example

Take $d = 2$. Let Animal $= [0, 10] \times [0, 10]$ and Dog $= [2, 5] \times [3, 7]$.

$$\text{Vol}(\text{Animal}) = 10 \times 10 = 100$$

$$\text{Vol}(\text{Dog}) = 3 \times 4 = 12$$

The intersection is Dog itself (Dog is inside Animal), so $\text{Vol}(\text{Animal} \cap \text{Dog}) = 12$.

$$s(\text{Animal}\mid\text{Dog}) = \frac{12}{12} = 1$$

$$s(\text{Dog}\mid\text{Animal}) = \frac{12}{100} = 0.12$$

Now add Cat $= [6, 9] \times [3, 7]$. Dog and Cat are disjoint (no overlap in the first coordinate: Dog’s $[2, 5]$ doesn’t intersect Cat’s $[6, 9]$), so $s(\text{Cat}\mid\text{Dog}) = 0$. Both are contained in Animal. The geometry directly encodes the taxonomy.

import numpy as np

def conditional_overlap(a_min, a_max, b_min, b_max):
    """Fraction of B's volume covered by A."""
    inter_min = np.maximum(a_min, b_min)
    inter_max = np.minimum(a_max, b_max)
    inter_sides = np.maximum(inter_max - inter_min, 0)
    b_sides = np.maximum(b_max - b_min, 1e-10)
    return np.prod(inter_sides) / np.prod(b_sides)

animal = (np.array([0, 0]), np.array([10, 10]))
dog    = (np.array([2, 3]), np.array([5, 7]))
cat    = (np.array([6, 3]), np.array([9, 7]))

print(conditional_overlap(*animal, *dog))  # s(Animal | Dog) = 1.0
print(conditional_overlap(*dog, *animal))  # s(Dog | Animal) = 0.12
print(conditional_overlap(*cat, *dog))     # s(Cat | Dog) = 0.0

In high dimensions, computing the raw volume $\prod_i (M_i - m_i)$ numerically is unstable: floating-point underflow to zero can occur before the full product is computed. Computing in log-space avoids that product: $\log \text{Vol}(B) = \sum_{i=1}^{d} \log(M_i - m_i)$. Practical implementations commonly use log-volumes for this reason.

# The Gradient Problem

Hard overlap-volume objectives have zero-gradient regions.When many pairs initialize disjoint, a naive overlap-volume objective can stall because those pairs supply no local direction of improvement. Consider two disjoint boxes – say Dog $= [2, 5] \times [3, 7]$ and Fish $= [20, 25] \times [30, 35]$. Their intersection volume is zero. If we move Dog slightly – say shift it by $\epsilon$ in any direction – the intersection is still zero. The loss doesn’t change. The gradient is zero.

Dasgupta et al. call this a local identifiability problem.In their definition, parameters are locally identifiable when every sufficiently small distinct perturbation changes the likelihood. Flat overlap-volume neighborhoods violate that criterion. The operational fact used here is the flat score, not a claim that every box-model parameter is globally unidentifiable. When boxes are disjoint, the containment probability is identically zero in a neighborhood of the current parameters. The optimizer receives no signal about which direction to move to bring the boxes closer.

The problem gets worse in high dimensions. In $\mathbb{R}^d$, two boxes are disjoint if any single coordinate has non-overlapping intervals. If independently initialized intervals overlap in one coordinate with probability $p<1$, then all coordinates overlap with probability $p^d$. The exact value depends on the initialization law, but the exponential dependence on $d$ makes fully overlapping random pairs scarce in high dimensions.

The intersection volume is piecewise multilinear in the box coordinates. Within each combinatorial regime (fully contained, partially overlapping, disjoint), it is a product of $d$ linear terms, one per coordinate. The containment probability, as a ratio of two such products, is a piecewise rational function. At the boundaries between regimes, the function is continuous but not differentiable. And in the disjoint regime, it is flat: identically zero, with zero gradient everywhere.

The failure mode is analogous to the dead ReLU problem in neural networks, but the mechanism differs. A dead ReLU neuron produces zero gradient for its current input but can recover if the bias shifts enough to re-enter the positive region. Here the problem is gradient sparsity at initialization: in high dimensions, almost every pair of boxes starts disjoint, so almost every training signal is zero. The gradient doesn’t die during training – it was never there to begin with. A dead ReLU affects one neuron; this affects all $4d$ parameters of every disjoint pair simultaneously.

Reproduce this figure

gradient_landscape.typ_diagram-style.typtypst compile gradient_landscape.typ gradient_landscape.png --ppi 500 --root ..

# First fix: smoothing (Li et al., 2019)

Li et al. 6 addressed the gradient problem by convolving the hard box indicator functions with Gaussian kernels. The smoothed intersection is never exactly zero; even for disjoint boxes, the Gaussian tails overlap, providing gradient signal.

The smoothed volume has a closed-form expression involving the Gaussian CDF, which is differentiable everywhere. This works: disjoint boxes now produce nonzero loss, and the optimizer can move them toward overlap.

The smoothed membership family is not closed under exact pointwise conjunction: multiplying two Gaussian-smoothed box indicators does not in general produce another member of the same parameterized family. That is a different statement from saying the underlying hard-box order vanished; the SmoothBox paper explicitly retains useful lattice properties. In the one-dimensional balanced-tree experiment reported by Dasgupta et al. 8, SmoothBox reached MRR 0.691 and Gumbel boxes 0.971. The paper attributes the gap to local identifiability and optimization. It is an empirical result in that setting, not a proved expressivity ceiling caused by “loss of lattice.”

# Second fix: Gumbel boxes (Dasgupta et al., 2020)

Dasgupta et al. 8 use an endpoint family whose min/max operations remain closed in distribution.

Instead of a deterministic box $[m_i, M_i]$, make each endpoint a random variable drawn from a Gumbel distribution. The Gumbel distribution appears as the limiting distribution of the maximum (or minimum) of many independent samples; it is to maxima what the Gaussian is to averages. It comes in two variants: Gumbel-max (right-skewed, models maxima) and Gumbel-min (left-skewed, models minima). The lower bound $m_i$ gets a Gumbel-max because box intersection takes the $\max$ of lower bounds; the upper bound $M_i$ gets a Gumbel-min because intersection takes the $\min$ of upper bounds.The pairing follows the intersection rule: the new lower bound is the maximum of the old lower bounds, while the new upper bound is the minimum of the old upper bounds.

$$m_i \sim \text{Gumbel}_{\max}(\mu^m_i, \beta), \qquad M_i \sim \text{Gumbel}_{\min}(\mu^M_i, \beta)$$

where $\mu^m_i, \mu^M_i$ are learnable location parameters and $\beta > 0$ is a temperature controlling the “softness” of the boundaries. At $\beta \to 0$, the Gumbel distributions collapse to point masses and we recover hard boxes.One analogy is wall thickness: a larger $\beta$ spreads the boundary transition, while a smaller $\beta$ sharpens it.

Why Gumbel specifically, and not Gaussian or logistic or any other smooth distribution? The answer is visible from the Gumbel-max CDF

$$F_\mu(x)=\exp\!\left[-e^{-(x-\mu)/\beta}\right].$$

For independent equal-scale variables $X_{1},\ldots,X_{k}$,

$$\begin{aligned} \Pr\!\left(\max_i X_i\leq x\right) &=\prod_i F_{\mu_i}(x)\\ &=\exp\!\left[-\sum_i e^{(\mu_i-x)/\beta}\right]\\ &=\exp\!\left[-e^{-(x-\mu_*)/\beta}\right], \end{aligned}$$

where

$$\mu_*=\beta\log\sum_i e^{\mu_i/\beta}.$$

The maximum is therefore another Gumbel-max variable, with its location updated by LogSumExp. Negating the variables gives the corresponding min-stability result. This is exactly the pair of operations needed for box intersection:

$$\max(X_1, \ldots, X_k) \sim \text{Gumbel}_{\max}\!\left(\beta \ln \sum_i e^{\mu_i / \beta}, \; \beta\right)$$

By the Fisher-Tippett-Gnedenko theorem, the only max-stable distribution families are Gumbel, Fréchet, and Weibull, so stability is not unique to Gumbel. The Gumbel construction combines it with the LogSumExp location above and a tractable expected-length integral.

Recall that box intersection takes the coordinate-wise max of lower bounds and min of upper bounds. With Gumbel endpoints, the intersection of two Gumbel boxes is again a Gumbel box, computed in closed form via LogSumExp. This is exact closure of the endpoint distribution family; the expected-volume score built on top of it is approximated.

# Where the Bessel function comes from

The expected side length of a Gumbel box along one coordinate involves a modified Bessel function $K_0$. What matters is its behavior: $K_0$ starts at infinity for argument zero, then drops off smoothly (monotonically decreasing, convex, and asymptotically $\sqrt{\pi / 2z} \, e^{-z}$). For large positive gap $\Delta_i$ between box endpoints (box is wide open), the expected volume is large. For $\Delta_i \approx 0$ (box is barely open), it is small but nonzero. For $\Delta_i < 0$ (endpoints are “inverted,” meaning the box is empty in expectation), it is tiny but still has a nonzero derivative $-K_1$. The optimizer retains a signal. Hard intersection volume lacks that signal; both SmoothBox and Gumbel boxes restore it with different constructions.

The formula: the expected side length is $\mathbb{E}[\max(M_i - m_i, 0)]$ where $M_i$ is Gumbel-min and $m_i$ is Gumbel-max. Integrating the product of a Gumbel-max PDF and a Gumbel-min survival function gives:

$$\mathbb{E}[\max(M_i - m_i, 0)] = 2\beta \, K_0\!\left(2 \, e^{-\Delta_i / (2\beta)}\right)$$

where $\Delta_i = \mu^M_i - \mu^m_i$ is the gap between the location parameters. In this calculation, a change of variables reduces the Gumbel endpoint integral to an integral representation of $K_0$.

For a random Gumbel box, the conditional-overlap quantity is itself a random ratio. The practical Gumbel-box score replaces $\mathbb E[V(A\cap B)/V(B)]$ with the ratio of expected volumes $\mathbb E[V(A\cap B)]/\mathbb E[V(B)]$. That is a deliberate tractable approximation, not an identity obtained from min/max stability.

A concrete comparison in 1D: two intervals that should overlap but are currently disjoint, with a gap of 2 units.

• Hard box: gradient zero everywhere. The optimizer has no direction.
• Softbox ($\sigma = 1$): Gaussian tails give a nonzero gradient. In the paper’s 1D balanced-tree experiment, MRR reached 0.691.
• Gumbel box ($\beta = 1$): the expected side-length calculation also gives a nonzero gradient. In the same experiment, MRR reached 0.971.

Gumbel boxes do more than change the smoothing kernel: min/max-stable endpoints keep intersections in the same distribution family while expected side lengths remain differentiable. The implemented score then layers the ratio-of-expectations and softplus approximations above that exact closure.

# The softplus approximation

Computing Bessel functions in every forward pass is expensive. Dasgupta et al. 8 observed that $2\beta K_0(2 e^{-x/(2\beta)})$ is nearly indistinguishable from a shifted softplus, and proposed:

$$2\beta \, K_0\!\left(2 \, e^{-\Delta_i/(2\beta)}\right) \approx \beta \, \log\!\left(1 + \exp\!\left(\frac{\Delta_i}{\beta} - 2\gamma\right)\right)$$

where $\gamma \approx 0.5772$ is the Euler-Mascheroni constant. Appendix C of Dasgupta et al. 8 reports a numerically observed maximum error of about $0.0617013\beta$ over $\Delta/\beta\in[-100,100]$. That is an empirical range check, not a proved global bound, and its practical effect depends on the surrounding loss and temperature.

The paper’s implementation uses this shifted softplus approximation. It is differentiable, monotonic with respect to containment, and numerically stable. The approximate expected volume is the product of the $d$ per-coordinate lengths. Its log is therefore a sum of log-softplus terms, plus the scale contribution $d\log\beta$, with the same linear-in-$d$ complexity as a hard box.

The construction now has several layers that should not be collapsed:

Reproduce this figure

gumbel_walls.typ_diagram-style.typtypst compile gumbel_walls.typ gumbel_walls.png --ppi 500 --root ..

The temperature $\beta$ can also be used as a curriculum: an implementation may start with softer boundaries and anneal toward a smaller value. That is a training strategy, not a requirement of the Gumbel-box derivation.

Gumbel boxes retain the union and complement limitations of hard boxes. Exact interval union would require the minimum of lower endpoints and maximum of upper endpoints, which are not the stable operations for the chosen endpoint families; a disjoint union is not one interval in any case. The complement of a Gumbel box is not a Gumbel box. Their contribution is differentiable expected volume while preserving the supported intersection operation.

# Query2Box: Answering Logical Queries

The first application branch asks whether boxes can do more than subsumption. Ren et al. 9 introduced Query2Box (the name says it: translate a logical query into a box), applying box embeddings to multi-hop logical queries over knowledge graphs. It uses learned query operators rather than the Gumbel expected-volume construction: represent a query as a box, then rank answer entities by distance to that box.

Consider the query: “Where did Canadian Turing Award winners graduate?”This is the running example from Ren et al. 9, and it also appears in Stanford CS224W lectures. This decomposes into: start with TuringAward (a point), apply the “Win” relation projection to get a box of Turing winners, separately start with Canada, project via “Citizen” to get a box of Canadians, intersect the two boxes, then project via “Graduate” to get universities. Each relation projection translates the box center and adds to the box offset, so the box can grow but not shrink. This parameterization accommodates one-to-many relations by expanding the candidate region.

Reproduce this figure

query2box_pipeline.typ_diagram-style.typtypst compile query2box_pipeline.typ query2box_pipeline.png --ppi 500 --root ..

Note the asymmetry: queries are boxes, but answer entities are still embedded as points.Query2Box uses points for candidate entities and boxes for queries; these representations serve different roles. The box represents the set of plausible answers; each candidate entity is a point that may or may not land inside it. The scoring function for a candidate answer entity $v$ (embedded as a point) relative to a query box $q$ is:

$$d(q, v) = d_{\text{out}}(q, v) + \alpha \cdot d_{\text{in}}(q, v)$$

where $d_{\text{out}}$ is the L1 distance from $v$ to the box itself (zero inside), while $d_{\text{in}}$ measures the distance from the box center to the coordinate-wise clamp of $v$ into the box. For an inside point the clamp is $v$; for an outside point it is the nearest point in the box. The hyperparameter $\alpha \in (0, 1)$, set to $0.2$ in the original paper, downweights this center-to-clamp term.

The asymmetry between $d_{\text{out}}$ and $d_{\text{in}}$ is intentional. Outside entities should be penalized heavily (they are not answers to the query). Inside entities are all plausible answers, but we mildly prefer those closer to the center.

Intersection of query boxes models conjunction: “countries that border France and have population > 50M” corresponds to intersecting two query boxes. Recall that geometric intersection is coordinate-wise max of lower bounds, min of upper bounds. This works when boxes overlap, but after multiple projection steps the boxes may not overlap at all, and an exact empty intersection gives zero volume with zero gradient (the same dead-zone problem from the previous section). Query2Box sidesteps this with a learned intersection operator: an attention mechanism over the input boxes’ centers produces the new center, combined with a coordinate-wise minimum of offsets to shrink the box. This is an approximation, not a geometric intersection, but it provides gradient signal in all configurations.

Two operations that boxes cannot handle:

Union. The union of two boxes is generally not a box. Query2Box works around this by transforming queries into disjunctive normal form (DNF): push all disjunctions to the last step, compute each conjunctive branch as a box, then aggregate scores across branches.DNF means rewriting the query so all ORs are at the outermost level. Each branch is a pure conjunction, representable as a single box. This is sound but adds computational cost proportional to the number of disjuncts.

Negation. The complement of a box in $\mathbb{R}^d$ is an unbounded region that is not a box. Queries like “European countries that do not border France” require a different approach. This limitation motivated two lines of follow-up work: geometric alternatives (cones) and algebraic alternatives (fuzzy logic).

# Beyond Boxes: Cones and Negation

Zhang et al. 12 introduced ConE (“cone embeddings”), which represents a query in $d$ dimensions as a Cartesian product $C_1\times\cdots\times C_d$. Each $C_i$ is a sector-cone on a circle, parameterized by an axis angle and an aperture angle.

The exact operation ConE gains is a particular negation: the closure-complement $\text{cl}(\mathbb R^2\setminus C)$ of a sector-cone is representable by flipping its axis and aperture. Ordinary set complement differs on the boundary. Intersections of sector-cones are not always sector-cones, so ConE learns a neural intersection approximation; the paper reports mean intersection Jaccard 0.6134 over 8,000 randomly sampled pairs of sector-cones.12 Union is handled after DNF as a set of branch cones, not collapsed to one cone.

Cones also have no finite volume, so they lack the finite-measure conditional score that boxes provide. That does not make box scores automatically calibrated: calibration is an empirical property to test with reliability curves, Brier score, or log loss. The geometries support different operations; neither one wins by definition.

BetaE 10 embeds queries as Beta distributions. Its learned negation operator maps each parameter pair by reciprocal, $(\alpha,\beta)\mapsto(1/\alpha,1/\beta)$. The t-norm fuzzy logic approach 11 13 decomposes complex queries into atomic link predictions aggregated with continuous AND/OR operators, without requiring training examples of the full complex queries.

# EL⁺⁺ and Ontology Completion

The second application branch returns to subsumption, but now asks how region constraints correspond to a formal ontology. Biomedical ontologies including SNOMED CT, Gene Ontology, and GALEN contain large fragments that can be normalized into lightweight description logics in the EL family. Embedding benchmarks often use an EL⁺⁺ normalization, a fragment of first-order logic designed for efficient reasoning.The source ontologies are not all written in exactly the same fragment. The benchmark normalization is the relevant object here. A classical reasoner derives logical consequences; an embedding model assigns scores that may be useful for ranking held-out or candidate axioms. EL⁺⁺ allows concept conjunction ($C \sqcap D$), existential restriction ($\exists r.C$, “things that have an $r$-relationship to some $C$”), and a bottom concept ($\bot$). A central inference is subsumption: given an ontology, determine whether $C \sqsubseteq D$ holds (every instance of $C$ is an instance of $D$).

Classical reasoners (ELK, Snorocket) compute subsumption exactly by rewriting the ontology’s axioms into normal forms: standardized shapes like $C \sqsubseteq D$ or $C_1 \sqcap C_2 \sqsubseteq D$ that decompose complex axioms into restricted forms. This is complete for logical entailment but does not rank plausible, unentailed subsumptions; embedding methods are used for that ranking task.

Jackermeier et al. 16 developed Box²EL (“dual box embeddings for EL”). It represents each role $r$ with a head box $\text{Head}(r)$ and a tail box $\text{Tail}(r)$, and gives each atomic concept $C$ a learned translation vector $\text{Bump}(C)$. The bump vectors let the role constraint depend on the concepts at both ends. Its geometric losses cover the four concept and existential forms below, along with separate losses for disjointness and role inclusions. The numbering here follows the Box²EL paper; some implementations, including subsume, swap NF1 and NF2.

NF1: $C \sqsubseteq D$. Direct subsumption. “Pneumonia ⊑ Disease-of-Respiratory-System” – every pneumonia is a respiratory disease. The Pneumonia box should sit inside the Disease-of-Respiratory-System box. In center-offset parameterization (storing a box as its midpoint $c$ and half-widths $o$ instead of min/max corners), define:

$$v = |c_C - c_D| + o_C - o_D.$$

Each coordinate $v_i$ is the larger one-sided boundary excess. Exact containment means $v_i \leq 0$ for every coordinate; the components need not equal zero. If centered box $C$ is wider than $D$ by $\delta$, then $C$ protrudes by $\delta/2$ on each side and $v_i = \delta/2$. One implementation-level loss is:

$$\mathcal{L}_{\text{NF1}} = \left\| \text{ReLU}\!\left(v - \epsilon\right) \right\|_2.$$

With $\epsilon = 0$, the loss is zero exactly when $C$ is contained in $D$. A positive $\epsilon$ tolerates up to $\epsilon$ of protrusion per coordinate; a negative value requires clearance inside $D$. Calling $\epsilon$ a margin does not by itself say which behavior is intended – its sign does.

NF2: $C_1 \sqcap C_2 \sqsubseteq D$. Conjunction implies subsumption. “Inflammation ⊓ Lung-Disorder ⊑ Pneumonia” – things that are both inflammatory and lung disorders are pneumonia. The intersection of the Inflammation box and the Lung-Disorder box should fit inside the Pneumonia box.

NF3: $C \sqsubseteq \exists r.D$. Existential restriction. “Pneumonia ⊑ ∃hasLocation.Lung” – every pneumonia has a location, and that location is a lung. Box²EL requires both

$$\text{Box}(C)+\text{Bump}(D) \subseteq\text{Head}(r) \quad\text{and}\quad \text{Box}(D)+\text{Bump}(C) \subseteq\text{Tail}(r).$$

NF4: $\exists r.C \sqsubseteq D$. Inverse restriction. “∃causedBy.Bacterium ⊑ Bacterial-Infection” – anything caused by a bacterium is a bacterial infection. In Box²EL, the possible sources connected to $C$ by $r$ lie within $\text{Head}(r)-\text{Bump}(C)$, so the loss requires

$$\text{Head}(r)-\text{Bump}(C) \subseteq\text{Box}(D).$$

The Box²EL evaluation compares against the ontology-embedding methods ELEm, EmEL⁺⁺, BoxEL, and ELBE on held-out subsumption axioms from GALEN, Gene Ontology, and Anatomy. The paper reports a median rank about 60% below the next-best method on GALEN, more than 80% below it on Gene Ontology, and more than 40% below it on Anatomy.16 These are results under that paper’s data splits and protocol, not a comparison with classical deductive reasoning.

# Volume as an order constraint

If the learned boxes satisfy $C\subseteq D$, then $\text{Vol}(C)\leq\text{Vol}(D)$ automatically. Along a correctly represented taxonomy, descendants therefore cannot be larger than their ancestors. This is a consequence of containment, not evidence that volume alone recovers semantic depth: unrelated concepts can have any relative volumes, and approximate training may violate the intended inclusions.

# Geometry Choices

The region-embedding literature developed along several connected lines: probabilistic boxes, multi-hop query answering, and ontology embeddings.

No single geometry is best across these tasks.The representation determines which operations can remain inside the model family; the optimizer then determines how the chosen constraints are fitted. Ontology-embedding objectives translate normalized axioms into geometric constraints, but zero loss guarantees only the conditions proved for that model. A geometry does not, by itself, force all and only the logical consequences of the source ontology. Graph-embedding approaches such as TransE and RotatE instead optimize scores on triples and do not supply a logical-model guarantee.

For subsumption-only tasks where inclusion is the chosen semantics, boxes or Gumbel boxes are a suitable model family; that choice does not by itself guarantee faithful ontology completion. Queries requiring negation or disjunction need an additional representation or decomposition, such as cones, DNF branches, or fuzzy operators. Hyperbolic cones (Ganea et al., 5) are the main alternative to boxes for hierarchies – hyperbolic space is a “continuous tree” where volume grows exponentially with radius, but a DAG with several independent parent chains is harder to place faithfully in one tree-like metric.Nickel & Kiela’s Poincaré embeddings (2017) do handle WordNet, which is a DAG. The limitation is not “DAGs are impossible”; it is that several unrelated hierarchies must share one metric geometry. Chami et al. (2020, MuRP) address relational variation with relation-specific parameters. Boxes can represent intersections directly: “things that are both mammals and flying creatures” is the overlap of two regions.

# Open Problems

Convexity. Helly-type intersection constraints impose real expressivity limits, but the statement depends on the region family and rule language. For ordinary axis-aligned boxes, pairwise intersection implies a common intersection because the intervals overlap in every coordinate. General convex sets in $\mathbb R^d$ instead have Helly number $d+1$, not 2. In a region-based rule setting, Charpenay and Schockaert 15 construct consistent combinations of hierarchy, intersection, and mutual-exclusion rules for which every convex embedding also satisfies an unintended hierarchy rule. That is a concrete counterexample to universal expressivity; it is not a claim that every ontology with multiple intersections is unrepresentable.

Faithfulness. Box embeddings can approximate but may not perfectly represent the logical structure of an ontology. BoxEL proves a local guarantee about its loss design: when the loss for an encoded axiom is zero, that axiom is satisfied by the induced geometric interpretation.14 This does not prove that every candidate subsumption scored by the model is entailed by the source ontology, nor that every entailed subsumption receives zero loss. Those are stronger no-extra-consequences and completeness requirements. TransBox (Yang et al., 20) keeps more EL⁺⁺ operations inside its representation and proves soundness properties, but not strong faithfulness for full EL⁺⁺.

FaithEL (Lacerda, Ozaki & Guimarães, 21) proves that normalized ELH has a strongly faithful convex embedding: the geometry can capture exactly the ontology’s consequences in that fragment. Extending comparable guarantees to richer role-composition settings remains an open direction. A pragmatic alternative, DELE (Mashkova et al., 17), computes deductive closure with a classical reasoner and uses it to train and evaluate the embedding without treating entailed statements as negatives.

Evaluation methodology. Negative-sampling protocols can draw axioms that are not asserted but are logically entailed, thereby penalizing a model for a true consequence. DELE 17 computes the deductive closure and filters it from the negative set. Papers also use different data splits and metrics, so cross-paper numerical comparisons require care.

Beyond axis-alignment. Axis-aligned boxes assume that dimensions are independent: the containment of Dog within Animal decomposes into $d$ independent interval containments. Octagons (Charpenay and Schockaert, 15) partially address this by adding diagonal constraints $x_i \pm x_j \le c$ at cost $O(d^2)$. Full covariance (oriented boxes, ellipsoids) would be more expressive, but the intersection of oriented boxes is not an oriented box. None of the practical families surveyed here is closed under intersection, union, complement, and projection.

# References

[1] Erk, K. (2009). “Representing Words as Regions in Vector Space.” CoNLL, 57–65. ↩

[2] Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J. & Yakhnenko, O. (2013). “Translating Embeddings for Modeling Multi-relational Data.” NeurIPS, 2787–2795. ↩

[3] Vendrov, I., Kiros, R., Fidler, S. & Urtasun, R. (2016). “Order-Embeddings of Images and Language.” ICLR. ↩

[4] Vilnis, L., Li, X., Murty, S. & McCallum, A. (2018). “Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures.” ACL, 263–272. ↩

[5] Ganea, O.-E., Becigneul, G. & Hofmann, T. (2018). “Hyperbolic Entailment Cones for Learning Hierarchical Embeddings.” ICML. ↩

[6] Li, X. L., Vilnis, L., Zhang, D., Boratko, M. & McCallum, A. (2019). “Smoothing the Geometry of Probabilistic Box Embeddings.” ICLR. ↩

[7] Sun, Z., Deng, Z.-H., Nie, J.-Y. & Tang, J. (2019). “RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space.” ICLR. ↩

[8] Dasgupta, S. S., Boratko, M., Zhang, D., Vilnis, L., Li, X. L. & McCallum, A. (2020). “Improving Local Identifiability in Probabilistic Box Embeddings.” NeurIPS. ↩

[9] Ren, H., Hu, W. & Leskovec, J. (2020). “Query2Box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings.” ICLR. ↩

[10] Ren, H. & Leskovec, J. (2020). “Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs.” NeurIPS. ↩

[11] Arakelyan, E., Daza, D., Minervini, P. & Cochez, M. (2021). “Complex Query Answering with Neural Link Predictors.” ICLR (Outstanding Paper). ↩

[12] Zhang, Z., Wang, J., Chen, J., Ji, S. & Wu, F. (2021). “ConE: Cone Embeddings for Multi-Hop Reasoning over Knowledge Graphs.” NeurIPS. ↩

[13] Chen, X., Hu, Z. & Sun, Y. (2022). “Fuzzy Logic Based Logical Query Answering on Knowledge Graphs.” AAAI. ↩

[14] Xiong, B., Potyka, N., Tran, T.-K., Nayyeri, M. & Staab, S. (2022). “Faithful Embeddings for EL⁺⁺ Knowledge Bases.” ECML-PKDD. ↩

[15] Charpenay, V. & Schockaert, S. (2024). “Embedding Ontologies with Octagons.” IJCAI. ↩

[16] Jackermeier, M., Chen, J. & Horrocks, I. (2024). “Dual Box Embeddings for the Description Logic EL⁺⁺.” WWW '24. ↩

[17] Mashkova, O., Zhapa-Camacho, F. & Hoehndorf, R. (2024). “DELE: Deductive EL⁺⁺ Embeddings for Knowledge Base Completion.” arXiv:2411.01574. ↩

[20] Yang, H., Chen, J. & Sattler, U. (2025). “TransBox: EL⁺⁺-closed Ontology Embedding.” WWW '25. ↩

[21] Lacerda, E., Ozaki, A. & Guimarães, R. (2024). “Strong Faithfulness for ELH Ontology Embeddings.” EKAW 2024. arXiv:2310.02198. ↩

[22] West, R., Gabrilovich, E., Murphy, K., Sun, S., Gupta, R. & Lin, D. (2014). “Knowledge Base Completion via Search-Based Question Answering.” WWW '14, 515–526. ↩