Variation de l'information

En théorie des probabilités et en théorie de l'information , la variation d'information , ou distance d'information partagée , mesure la distance entre deux regroupements ( partitions d'éléments ). Elle est étroitement liée à l'information mutuelle ; en effet, il s'agit d'une expression linéaire simple faisant intervenir l'information mutuelle. Contrairement à cette dernière, la variation d'information est une véritable métrique , car elle satisfait l' inégalité triangulaire .

Définition

Supposons que nous ayons deux partitions ${\displaystyle X <semantics><mrow><mstyle><mi>X</mi></mstyle></mrow><annotation>{\displaystyle X}</annotation></semantics></math><img src=$ ${\displaystyle Y <semantics><mrow><mstyle><mi>Y</mi></mstyle></mrow><annotation>{\displaystyle Y}</annotation></semantics></math><img src=$ ${\displaystyle A <semantics><mrow><mstyle><mi>UN</mi></mstyle></mrow><annotation>{\displaystyle A}</annotation></semantics></math><img src=$ ${\displaystyle X=\{X_{1},X_{2},\ldots ,X_{k}\ <semantics><mrow><mstyle><mi>X</mi><mo>=</mo><mo>{</mo><msub><mi>X</mi><mrow><mn>1</mn></mrow></msub><mo>,</mo><msub><mi>X</mi><mrow><mn>2</mn></mrow></msub><mo>,</mo><mo>…</mo><mo>,</mo><msub><mi>X</mi><mrow><mi>k</mi></mrow></msub><mo>}</mo></mstyle></mrow><annotation>{\displaystyle X=\{X_{1},X_{2},\ldots ,X_{k}\}}</annotation></semantics></math><img src=$ ${\displaystyle Y=\{Y_{1},Y_{2},\ldots ,Y_{l}\ <semantics><mrow><mstyle><mi>Y</mi><mo>=</mo><mo>{</mo><msub><mi>Y</mi><mrow><mn>1</mn></mrow></msub><mo>,</mo><msub><mi>Y</mi><mrow><mn>2</mn></mrow></msub><mo>,</mo><mo>…</mo><mo>,</mo><msub><mi>Y</mi><mrow><mi>l</mi></mrow></msub><mo>}</mo></mstyle></mrow><annotation>{\displaystyle Y=\{Y_{1},Y_{2},\ldots ,Y_{l}\}}</annotation></semantics></math><img src=$

Laisser:

{\displaystyle n=\sum _{i}|X_{i}|=\sum _{j}|Y_{j}|=|A| <semantics><mrow><mstyle><mi>n</mi><mo>=</mo><munder><mo>∑</mo><mrow><mi>je</mi></mrow></munder><mrow><mo>|</mo></mrow><msub><mi>X</mi><mrow><mi>je</mi></mrow></msub><mrow><mo>|</mo></mrow><mo>=</mo><munder><mo>∑</mo><mrow><mi>j</mi></mrow></munder><mrow><mo>|</mo></mrow><msub><mi>Y</mi><mrow><mi>j</mi></mrow></msub><mrow><mo>|</mo></mrow><mo>=</mo><mrow><mo>|</mo></mrow><mi>UN</mi><mrow><mo>|</mo></mrow></mstyle></mrow><annotation>{\displaystyle n=\sum _{i}|X_{i}|=\sum _{j}|Y_{j}|=|A|}</annotation></semantics></math></span><img src=

{\displaystyle p_{i}=|X_{i}|/n <semantics><mrow><mstyle><msub><mi>p</mi><mrow><mi>je</mi></mrow></msub><mo>=</mo><mrow><mo>|</mo></mrow><msub><mi>X</mi><mrow><mi>je</mi></mrow></msub><mrow><mo>|</mo></mrow><mrow><mo>/</mo></mrow><mi>n</mi></mstyle></mrow><annotation>{\displaystyle p_{i}=|X_{i}|/n}</annotation></semantics></math></span><img src=

{\displaystyle q_{j}=|Y_{j}|/n <semantics><mrow><mstyle><msub><mi>q</mi><mrow><mi>j</mi></mrow></msub><mo>=</mo><mrow><mo>|</mo></mrow><msub><mi>Y</mi><mrow><mi>j</mi></mrow></msub><mrow><mo>|</mo></mrow><mrow><mo>/</mo></mrow><mi>n</mi></mstyle></mrow><annotation>{\displaystyle q_{j}=|Y_{j}|/n}</annotation></semantics></math></span><img src=

{\displaystyle r_{ij}=|X_{i}\cap Y_{j}|/n <semantics><mrow><mstyle><msub><mi>r</mi><mrow><mi>je</mi><mi>j</mi></mrow></msub><mo>=</mo><mrow><mo>|</mo></mrow><msub><mi>X</mi><mrow><mi>je</mi></mrow></msub><mo>∩</mo><msub><mi>Y</mi><mrow><mi>j</mi></mrow></msub><mrow><mo>|</mo></mrow><mrow><mo>/</mo></mrow><mi>n</mi></mstyle></mrow><annotation>{\displaystyle r_{ij}=|X_{i}\cap Y_{j}|/n}</annotation></semantics></math></span><img src=

La variation d'information entre les deux partitions est alors la suivante :

{\displaystyle \mathrm {VI} (X;Y)=-\sum _{i,j}r_{ij}\left[\log(r_{ij}/p_{i})+\log(r_{ij}/q_{j}) ight] <semantics><mrow><mstyle><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo>)</mo><mo>=</mo><mo>−</mo><munder><mo>∑</mo><mrow><mi>je</mi><mo>,</mo><mi>j</mi></mrow></munder><msub><mi>r</mi><mrow><mi>je</mi><mi>j</mi></mrow></msub><mrow><mo>[</mo><mrow><mi>enregistrer</mi><mo>⁡</mo><mo>(</mo><msub><mi>r</mi><mrow><mi>je</mi><mi>j</mi></mrow></msub><mrow><mo>/</mo></mrow><msub><mi>p</mi><mrow><mi>je</mi></mrow></msub><mo>)</mo><mo>+</mo><mi>enregistrer</mi><mo>⁡</mo><mo>(</mo><msub><mi>r</mi><mrow><mi>je</mi><mi>j</mi></mrow></msub><mrow><mo>/</mo></mrow><msub><mi>q</mi><mrow><mi>j</mi></mrow></msub><mo>)</mo></mrow><mo>]</mo></mrow></mstyle></mrow><annotation>{\displaystyle \mathrm {VI} (X;Y)=-\sum _{i,j}r_{ij}\left[\log(r_{ij}/p_{i})+\log(r_{ij}/q_{j}) ight]}</annotation></semantics></math></span><img src=

Cela équivaut à la distance d'information partagée entre les variables aléatoires i et j par rapport à la mesure de probabilité uniforme sur ${\displaystyle A <semantics><mrow><mstyle><mi>UN</mi></mstyle></mrow><annotation>{\displaystyle A}</annotation></semantics></math><img src=$ ${\displaystyle \mu (B):=|B|/n <semantics><mrow><mstyle><mi>μ</mi><mo>(</mo><mi>B</mi><mo>)</mo><mo>:=</mo><mrow><mo>|</mo></mrow><mi>B</mi><mrow><mo>|</mo></mrow><mrow><mo>/</mo></mrow><mi>n</mi></mstyle></mrow><annotation>{\displaystyle \mu (B):=|B|/n}</annotation></semantics></math><img src=$ ${\displaystyle B\subseteq A <semantics><mrow><mstyle><mi>B</mi><mo>⊆</mo><mi>UN</mi></mstyle></mrow><annotation>{\displaystyle B\subseteq A}</annotation></semantics></math><img src=$

Contenu d'information explicite

Nous pouvons reformuler cette définition en des termes qui mettent explicitement en évidence le contenu informationnel de cette métrique.

L'ensemble de toutes les partitions d'un ensemble forme un treillis compact où l'ordre partiel induit deux opérations, la rencontre ${\displaystyle \wedge <semantics><mrow><mstyle><mo>∧</mo></mstyle></mrow><annotation>{\displaystyle \wedge }</annotation></semantics></math><img src=$ ${\displaystyle \vee <semantics><mrow><mstyle><mo>∨</mo></mstyle></mrow><annotation>{\displaystyle \vee }</annotation></semantics></math><img src=$ ${\displaystyle {\overline {\mathrm {1} <semantics><mrow><mstyle><mrow><mover><mrow><mn>1</mn></mrow><mo>¯</mo></mover></mrow></mstyle></mrow><annotation>{\displaystyle {\overline {\mathrm {1} }}}</annotation></semantics></math><img src=$ ${\displaystyle {\overline {\mathrm {0} <semantics><mrow><mstyle><mrow><mover><mrow><mn>0</mn></mrow><mo>¯</mo></mover></mrow></mstyle></mrow><annotation>{\displaystyle {\overline {\mathrm {0} }}}</annotation></semantics></math><img src=$ ${\displaystyle X <semantics><mrow><mstyle><mi>X</mi></mstyle></mrow><annotation>{\displaystyle X}</annotation></semantics></math><img src=$ ${\displaystyle Y <semantics><mrow><mstyle><mi>Y</mi></mstyle></mrow><annotation>{\displaystyle Y}</annotation></semantics></math><img src=$ ${\displaystyle X_{i <semantics><mrow><mstyle><msub><mi>X</mi><mrow><mi>je</mi></mrow></msub></mstyle></mrow><annotation>{\displaystyle X_{i}}</annotation></semantics></math><img src=$ ${\displaystyle X <semantics><mrow><mstyle><mi>X</mi></mstyle></mrow><annotation>{\displaystyle X}</annotation></semantics></math><img src=$ ${\displaystyle Y_{i <semantics><mrow><mstyle><msub><mi>Y</mi><mrow><mi>je</mi></mrow></msub></mstyle></mrow><annotation>{\displaystyle Y_{i}}</annotation></semantics></math><img src=$ ${\displaystyle Y <semantics><mrow><mstyle><mi>Y</mi></mstyle></mrow><annotation>{\displaystyle Y}</annotation></semantics></math><img src=$ ${\displaystyle X\wedge Y\subseteq X <semantics><mrow><mstyle><mi>X</mi><mo>∧</mo><mi>Y</mi><mo>⊆</mo><mi>X</mi></mstyle></mrow><annotation>{\displaystyle X\wedge Y\subseteq X}</annotation></semantics></math><img src=$ ${\displaystyle X\wedge Y\subseteq Y <semantics><mrow><mstyle><mi>X</mi><mo>∧</mo><mi>Y</mi><mo>⊆</mo><mi>Y</mi></mstyle></mrow><annotation>{\displaystyle X\wedge Y\subseteq Y}</annotation></semantics></math><img src=$

Définissons l'entropie d'une partition ${\displaystyle X <semantics><mrow><mstyle><mi>X</mi></mstyle></mrow><annotation>{\displaystyle X}</annotation></semantics></math><img src=$

{\displaystyle H\left(X ight)\,=\,-\sum _{i}\,p_{i}\log p_{i <semantics><mrow><mstyle><mi>H</mi><mrow><mo>(</mo><mi>X</mi><mo>)</mo></mrow><mspace></mspace><mo>=</mo><mspace></mspace><mo>−</mo><munder><mo>∑</mo><mrow><mi>je</mi></mrow></munder><mspace></mspace><msub><mi>p</mi><mrow><mi>je</mi></mrow></msub><mi>enregistrer</mi><mo>⁡</mo><msub><mi>p</mi><mrow><mi>je</mi></mrow></msub></mstyle></mrow><annotation>{\displaystyle H\left(X ight)\,=\,-\sum _{i}\,p_{i}\log p_{i}}</annotation></semantics></math></span><img src=

où ${\displaystyle p_{i}=|X_{i}|/n <semantics><mrow><mstyle><msub><mi>p</mi><mrow><mi>je</mi></mrow></msub><mo>=</mo><mrow><mo>|</mo></mrow><msub><mi>X</mi><mrow><mi>je</mi></mrow></msub><mrow><mo>|</mo></mrow><mrow><mo>/</mo></mrow><mi>n</mi></mstyle></mrow><annotation>{\displaystyle p_{i}=|X_{i}|/n}</annotation></semantics></math><img src=$ ${\displaystyle H({\overline {\mathrm {1} }})=0 <semantics><mrow><mstyle><mi>H</mi><mo>(</mo><mrow><mover><mrow><mn>1</mn></mrow><mo>¯</mo></mover></mrow><mo>)</mo><mo>=</mo><mn>0</mn></mstyle></mrow><annotation>{\displaystyle H({\overline {\mathrm {1} }})=0}</annotation></semantics></math><img src=$ ${\displaystyle H({\overline {\mathrm {0} }})=\log \,n <semantics><mrow><mstyle><mi>H</mi><mo>(</mo><mrow><mover><mrow><mn>0</mn></mrow><mo>¯</mo></mover></mrow><mo>)</mo><mo>=</mo><mi>enregistrer</mi><mspace></mspace><mi>n</mi></mstyle></mrow><annotation>{\displaystyle H({\overline {\mathrm {0} }})=\log \,n}</annotation></semantics></math><img src=$ ${\displaystyle X\subseteq Y\Rightarrow H(X)\geq H(Y) <semantics><mrow><mstyle><mi>X</mi><mo>⊆</mo><mi>Y</mi><mo>⇒</mo><mi>H</mi><mo>(</mo><mi>X</mi><mo>)</mo><mo>≥</mo><mi>H</mi><mo>(</mo><mi>Y</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle X\subseteq Y\Rightarrow H(X)\geq H(Y)}</annotation></semantics></math><img src=$

Ensuite, la distance VI entre ${\displaystyle X <semantics><mrow><mstyle><mi>X</mi></mstyle></mrow><annotation>{\displaystyle X}</annotation></semantics></math><img src=$ ${\displaystyle Y <semantics><mrow><mstyle><mi>Y</mi></mstyle></mrow><annotation>{\displaystyle Y}</annotation></semantics></math><img src=$

{\displaystyle \mathrm {VI} (X,Y)\,=\,2H(X\wedge Y)\,-\,H(X)\,-\,H(Y) <semantics><mrow><mstyle><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo>)</mo><mspace></mspace><mo>=</mo><mspace></mspace><mn>2</mn><mi>H</mi><mo>(</mo><mi>X</mi><mo>∧</mo><mi>Y</mi><mo>)</mo><mspace></mspace><mo>−</mo><mspace></mspace><mi>H</mi><mo>(</mo><mi>X</mi><mo>)</mo><mspace></mspace><mo>−</mo><mspace></mspace><mi>H</mi><mo>(</mo><mi>Y</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle \mathrm {VI} (X,Y)\,=\,2H(X\wedge Y)\,-\,H(X)\,-\,H(Y)}</annotation></semantics></math></span><img src=

La différence ${\displaystyle d(X,Y)\equiv |H\left(X ight)-H\left(Y ight)| <semantics><mrow><mstyle><mi>d</mi><mo>(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo>)</mo><mo>≡</mo><mrow><mo>|</mo></mrow><mi>H</mi><mrow><mo>(</mo><mi>X</mi><mo>)</mo></mrow><mo>−</mo><mi>H</mi><mrow><mo>(</mo><mi>Y</mi><mo>)</mo></mrow><mrow><mo>|</mo></mrow></mstyle></mrow><annotation>{\displaystyle d(X,Y)\equiv |H\left(X ight)-H\left(Y ight)|}</annotation></semantics></math><img src=$ ${\displaystyle d(X,Y)=0 <semantics><mrow><mstyle><mi>d</mi><mo>(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo>)</mo><mo>=</mo><mn>0</mn></mstyle></mrow><annotation>{\displaystyle d(X,Y)=0}</annotation></semantics></math><img src=$ ${\displaystyle X=Y <semantics><mrow><mstyle><mi>X</mi><mo>=</mo><mi>Y</mi></mstyle></mrow><annotation>{\displaystyle X=Y}</annotation></semantics></math><img src=$ ${\displaystyle {\overline {\mathrm {1} <semantics><mrow><mstyle><mrow><mover><mrow><mn>1</mn></mrow><mo>¯</mo></mover></mrow></mstyle></mrow><annotation>{\displaystyle {\overline {\mathrm {1} }}}</annotation></semantics></math><img src=$ ${\displaystyle \mathrm {VI} (X,\mathrm {1} )\,=\,H\left(X ight) <semantics><mrow><mstyle><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>,</mo><mrow><mn>1</mn></mrow><mo>)</mo><mspace></mspace><mo>=</mo><mspace></mspace><mi>H</mi><mrow><mo>(</mo><mi>X</mi><mo>)</mo></mrow></mstyle></mrow><annotation>{\displaystyle \mathrm {VI} (X,\mathrm {1} )\,=\,H\left(X ight)}</annotation></semantics></math><img src=$

Si, dans le diagramme de Hasse, nous traçons une arête de chaque partition vers le maximum ${\displaystyle {\overline {\mathrm {1} <semantics><mrow><mstyle><mrow><mover><mrow><mn>1</mn></mrow><mo>¯</mo></mover></mrow></mstyle></mrow><annotation>{\displaystyle {\overline {\mathrm {1} }}}</annotation></semantics></math><img src=$ ${\displaystyle {\overline {\mathrm {1} <semantics><mrow><mstyle><mrow><mover><mrow><mn>1</mn></mrow><mo>¯</mo></mover></mrow></mstyle></mrow><annotation>{\displaystyle {\overline {\mathrm {1} }}}</annotation></semantics></math><img src=$

{\displaystyle \mathrm {VI} (X,Y)\,=\,|\mathrm {VI} (X,{\overline {\mathrm {1} }})\,-\,\mathrm {VI} (X\wedge Y,{\overline {\mathrm {1} }})|\,+\,|\mathrm {VI} (Y,{\overline {\mathrm {1} }})\,-\,\mathrm {VI} (X\wedge Y,{\overline {\mathrm {1} }})|\,=\,d(X,X\wedge Y)\,+\,d(Y,X\wedge Y) <semantics><mrow><mstyle><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo>)</mo><mspace></mspace><mo>=</mo><mspace></mspace><mrow><mo>|</mo></mrow><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>,</mo><mrow><mover><mrow><mn>1</mn></mrow><mo>¯</mo></mover></mrow><mo>)</mo><mspace></mspace><mo>−</mo><mspace></mspace><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>∧</mo><mi>Y</mi><mo>,</mo><mrow><mover><mrow><mn>1</mn></mrow><mo>¯</mo></mover></mrow><mo>)</mo><mrow><mo>|</mo></mrow><mspace></mspace><mo>+</mo><mspace></mspace><mrow><mo>|</mo></mrow><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>Y</mi><mo>,</mo><mrow><mover><mrow><mn>1</mn></mrow><mo>¯</mo></mover></mrow><mo>)</mo><mspace></mspace><mo>−</mo><mspace></mspace><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>∧</mo><mi>Y</mi><mo>,</mo><mrow><mover><mrow><mn>1</mn></mrow><mo>¯</mo></mover></mrow><mo>)</mo><mrow><mo>|</mo></mrow><mspace></mspace><mo>=</mo><mspace></mspace><mi>d</mi><mo>(</mo><mi>X</mi><mo>,</mo><mi>X</mi><mo>∧</mo><mi>Y</mi><mo>)</mo><mspace></mspace><mo>+</mo><mspace></mspace><mi>d</mi><mo>(</mo><mi>Y</mi><mo>,</mo><mi>X</mi><mo>∧</mo><mi>Y</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle \mathrm {VI} (X,Y)\,=\,|\mathrm {VI} (X,{\overline {\mathrm {1} }})\,-\,\mathrm {VI} (X\wedge Y,{\overline {\mathrm {1} }})|\,+\,|\mathrm {VI} (Y,{\overline {\mathrm {1} }})\,-\,\mathrm {VI} (X\wedge Y,{\overline {\mathrm {1} }})|\,=\,d(X,X\wedge Y)\,+\,d(Y,X\wedge Y)}</annotation></semantics></math></span><img src=

Pour ${\displaystyle H(X) <semantics><mrow><mstyle><mi>H</mi><mo>(</mo><mi>X</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle H(X)}</annotation></semantics></math><img src=$

{\displaystyle H(X,Y)\,=\,H(X\wedge Y) <semantics><mrow><mstyle><mi>H</mi><mo>(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo>)</mo><mspace></mspace><mo>=</mo><mspace></mspace><mi>H</mi><mo>(</mo><mi>X</mi><mo>∧</mo><mi>Y</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle H(X,Y)\,=\,H(X\wedge Y)}</annotation></semantics></math></span><img src=

et nous avons aussi cela ${\displaystyle d(X,X\wedge Y)\,=\,H(X\wedge Y|X) <semantics><mrow><mstyle><mi>d</mi><mo>(</mo><mi>X</mi><mo>,</mo><mi>X</mi><mo>∧</mo><mi>Y</mi><mo>)</mo><mspace></mspace><mo>=</mo><mspace></mspace><mi>H</mi><mo>(</mo><mi>X</mi><mo>∧</mo><mi>Y</mi><mrow><mo>|</mo></mrow><mi>X</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle d(X,X\wedge Y)\,=\,H(X\wedge Y|X)}</annotation></semantics></math><img src=$ ${\displaystyle X\wedge Y <semantics><mrow><mstyle><mi>X</mi><mo>∧</mo><mi>Y</mi></mstyle></mrow><annotation>{\displaystyle X\wedge Y}</annotation></semantics></math><img src=$ ${\displaystyle X <semantics><mrow><mstyle><mi>X</mi></mstyle></mrow><annotation>{\displaystyle X}</annotation></semantics></math><img src=$

Identités

La variation de l'information satisfait

{\displaystyle \mathrm {VI} (X;Y)=H(X)+H(Y)-2I(X,Y) <semantics><mrow><mstyle><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo>)</mo><mo>=</mo><mi>H</mi><mo>(</mo><mi>X</mi><mo>)</mo><mo>+</mo><mi>H</mi><mo>(</mo><mi>Y</mi><mo>)</mo><mo>−</mo><mn>2</mn><mi>je</mi><mo>(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle \mathrm {VI} (X;Y)=H(X)+H(Y)-2I(X,Y)}</annotation></semantics></math></span><img src=

où ${\displaystyle H(X) <semantics><mrow><mstyle><mi>H</mi><mo>(</mo><mi>X</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle H(X)}</annotation></semantics></math><img src=$ ${\displaystyle X <semantics><mrow><mstyle><mi>X</mi></mstyle></mrow><annotation>{\displaystyle X}</annotation></semantics></math><img src=$ ${\displaystyle I(X,Y) <semantics><mrow><mstyle><mi>je</mi><mo>(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle I(X,Y)}</annotation></semantics></math><img src=$ ${\displaystyle X <semantics><mrow><mstyle><mi>X</mi></mstyle></mrow><annotation>{\displaystyle X}</annotation></semantics></math><img src=$ ${\displaystyle Y <semantics><mrow><mstyle><mi>Y</mi></mstyle></mrow><annotation>{\displaystyle Y}</annotation></semantics></math><img src=$ ${\displaystyle A <semantics><mrow><mstyle><mi>UN</mi></mstyle></mrow><annotation>{\displaystyle A}</annotation></semantics></math><img src=$

{\displaystyle \mathrm {VI} (X;Y)=H(X,Y)-I(X,Y) <semantics><mrow><mstyle><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo>)</mo><mo>=</mo><mi>H</mi><mo>(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo>)</mo><mo>−</mo><mi>je</mi><mo>(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle \mathrm {VI} (X;Y)=H(X,Y)-I(X,Y)}</annotation></semantics></math></span><img src=

où ${\displaystyle H(X,Y) <semantics><mrow><mstyle><mi>H</mi><mo>(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle H(X,Y)}</annotation></semantics></math><img src=$ ${\displaystyle X <semantics><mrow><mstyle><mi>X</mi></mstyle></mrow><annotation>{\displaystyle X}</annotation></semantics></math><img src=$ ${\displaystyle Y <semantics><mrow><mstyle><mi>Y</mi></mstyle></mrow><annotation>{\displaystyle Y}</annotation></semantics></math><img src=$

{\displaystyle \mathrm {VI} (X;Y)=H(X|Y)+H(Y|X) <semantics><mrow><mstyle><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo>)</mo><mo>=</mo><mi>H</mi><mo>(</mo><mi>X</mi><mrow><mo>|</mo></mrow><mi>Y</mi><mo>)</mo><mo>+</mo><mi>H</mi><mo>(</mo><mi>Y</mi><mrow><mo>|</mo></mrow><mi>X</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle \mathrm {VI} (X;Y)=H(X|Y)+H(Y|X)}</annotation></semantics></math></span><img src=

où ${\displaystyle H(X|Y) <semantics><mrow><mstyle><mi>H</mi><mo>(</mo><mi>X</mi><mrow><mo>|</mo></mrow><mi>Y</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle H(X|Y)}</annotation></semantics></math><img src=$ ${\displaystyle H(Y|X) <semantics><mrow><mstyle><mi>H</mi><mo>(</mo><mi>Y</mi><mrow><mo>|</mo></mrow><mi>X</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle H(Y|X)}</annotation></semantics></math><img src=$

La variation de l'information peut également être limitée, soit en termes de nombre d'éléments :

{\displaystyle \mathrm {VI} (X;Y)\leq \log(n) <semantics><mrow><mstyle><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo>)</mo><mo>≤</mo><mi>enregistrer</mi><mo>⁡</mo><mo>(</mo><mi>n</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle \mathrm {VI} (X;Y)\leq \log(n)}</annotation></semantics></math></span><img src=

Ou, en ce qui concerne un nombre maximal de clusters, ${\displaystyle K^{* <semantics><mrow><mstyle><msup><mi>K</mi><mrow><mo>∗</mo></mrow></msup></mstyle></mrow><annotation>{\displaystyle K^{*}}</annotation></semantics></math><img src=$

{\displaystyle \mathrm {VI} (X;Y)\leq 2\log(K^{*}) <semantics><mrow><mstyle><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo>)</mo><mo>≤</mo><mn>2</mn><mi>enregistrer</mi><mo>⁡</mo><mo>(</mo><msup><mi>K</mi><mrow><mo>∗</mo></mrow></msup><mo>)</mo></mstyle></mrow><annotation>{\displaystyle \mathrm {VI} (X;Y)\leq 2\log(K^{*})}</annotation></semantics></math></span><img src=

Inégalité triangulaire

Pour vérifier l'inégalité triangulaire ${\displaystyle \mathrm {VI} (X;Z)\leq \mathrm {VI} (X;Y)+\mathrm {VI} (Y;Z) <semantics><mrow><mstyle><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>;</mo><mi>Z</mi><mo>)</mo><mo>≤</mo><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo>)</mo><mo>+</mo><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>Y</mi><mo>;</mo><mi>Z</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle \mathrm {VI} (X;Z)\leq \mathrm {VI} (X;Y)+\mathrm {VI} (Y;Z)}</annotation></semantics></math><img src=$ ${\displaystyle \mathrm {VI} (X;Y)=H(X|Y)+H(Y|X) <semantics><mrow><mstyle><mrow><mi>V</mi><mi>je</mi></mrow><mo>(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo>)</mo><mo>=</mo><mi>H</mi><mo>(</mo><mi>X</mi><mrow><mo>|</mo></mrow><mi>Y</mi><mo>)</mo><mo>+</mo><mi>H</mi><mo>(</mo><mi>Y</mi><mrow><mo>|</mo></mrow><mi>X</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle \mathrm {VI} (X;Y)=H(X|Y)+H(Y|X)}</annotation></semantics></math><img src=$ ${\displaystyle H(X|Z)\leq H(X|Y)+H(Y|Z) <semantics><mrow><mstyle><mi>H</mi><mo>(</mo><mi>X</mi><mrow><mo>|</mo></mrow><mi>Z</mi><mo>)</mo><mo>≤</mo><mi>H</mi><mo>(</mo><mi>X</mi><mrow><mo>|</mo></mrow><mi>Y</mi><mo>)</mo><mo>+</mo><mi>H</mi><mo>(</mo><mi>Y</mi><mrow><mo>|</mo></mrow><mi>Z</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle H(X|Z)\leq H(X|Y)+H(Y|Z)}</annotation></semantics></math><img src=$ ${\displaystyle H(X|Y)+H(Y|Z)\geq H(X|Y,Z)+H(Y|Z)=H(X,Y|Z) <semantics><mrow><mstyle><mi>H</mi><mo>(</mo><mi>X</mi><mrow><mo>|</mo></mrow><mi>Y</mi><mo>)</mo><mo>+</mo><mi>H</mi><mo>(</mo><mi>Y</mi><mrow><mo>|</mo></mrow><mi>Z</mi><mo>)</mo><mo>≥</mo><mi>H</mi><mo>(</mo><mi>X</mi><mrow><mo>|</mo></mrow><mi>Y</mi><mo>,</mo><mi>Z</mi><mo>)</mo><mo>+</mo><mi>H</mi><mo>(</mo><mi>Y</mi><mrow><mo>|</mo></mrow><mi>Z</mi><mo>)</mo><mo>=</mo><mi>H</mi><mo>(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mrow><mo>|</mo></mrow><mi>Z</mi><mo>)</mo></mstyle></mrow><annotation>{\displaystyle H(X|Y)+H(Y|Z)\geq H(X|Y,Z)+H(Y|Z)=H(X,Y|Z)}</annotation></semantics></math><img src=$

Meila, M. (2007). « Comparaison de regroupements — une distance basée sur l’information » . Journal of Multivariate Analysis . 98 (5) : 873–895 . doi : 10.1016/j.jmva.2006.11.013 .
Kingsford, Carl (2009). « Notes sur la théorie de l’information » (PDF) 2009 .
Kraskov, Alexander; Harald Stögbauer; Ralph G. Andrzejak; Peter Grassberger (2003). « Clustering hiérarchique basé sur l’information mutuelle ». arXiv : q-bio/0311039 .