Entropy, order and an “operational taxonomy of order”

An extended and reasoned synthesis of results obtained on particle systems, between Shannon entropy, spectral order, orientational order and different families of structure.

Author: Manuele Turini
License: This work is distributed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).
Table of contents

1. Starting point: entropy and internal organization

The fundamental question from which this work stems is simple to formulate yet very deep from a physical perspective: is disorder truly a single quantity? Or, more precisely, is what we call “order” or “disorder” actually a set of distinct phenomena that cannot be compressed into a single number?

In statistical mechanics, classical entropy measures the multiplicity of microstates compatible with a macrostate. This information is fundamental, but it does not tell the whole story: it does not distinguish how microstates are organized in space, nor whether they contain local symmetries, clusters, partial periodicity, short-range exclusion or emergent collective structures.

To grasp the problem, consider two sets of particles with the same average density and the same total number of accessible configurations. One may appear almost random, the other may exhibit clustering, domains, local regularities or even a perturbed lattice. From the “combinatorial” entropy perspective, the two cases can be close; from a structural perspective, however, they are clearly different.

Core idea: entropy measures the richness of possible states; order, on the other hand, describes the geometry, the correlation and the internal coherence of those states. They are connected, but not identical.

2. Boltzmann, Gibbs and Shannon entropy: analogies and limits

2.1 Boltzmann entropy

The most well-known form is

S = k_B ln W

where W is the number of microstates compatible with the observed macrostate and k_B is the Boltzmann constant. This formula relates entropy to the number of possible configurations. The larger W, the higher the entropy.

This definition is extremely powerful, but it operates on a global level: it does not automatically see the difference between “geometrically dispersed” microstates and “geometrically organized” microstates, unless that difference is reflected in the count of W itself.

2.2 Gibbs entropy

When microstates are not equally probable, the more general form is

S = -k_B Σ_i p_i ln p_i

where p_i is the probability of microstate i. This formulation brings physical entropy closer to information entropy. If all microstates are equally probable, i.e., p_i = 1/W, we immediately recover Boltzmann's formula.

2.3 Shannon entropy

In information theory, the formal analogue is

H = - Σ_i p_i log p_i

which measures the uncertainty of a source or a distribution. Using base-2 logarithm yields bits; using the natural logarithm yields nats.

Shannon entropy is perfect for quantifying statistical dispersion, unpredictability and average information content. However, exactly like Gibbs entropy, it is not automatically a measure of shape or topology. Two distributions with similar entropy can correspond to very different spatial structures.

2.4 The crucial point

The theoretical issue that emerged during the work is therefore: entropy is sensitive to the distribution of probabilities, but not always to the geometric nature of correlations. To capture internal order, additional observables are needed.

This does not mean entropy is “wrong”; it means it answers a different question. Entropy asks: “how spread out or numerous is the possibility?”. Order asks: “what internal architecture do the configurations possess?”

3. Why “order” does not coincide with “low entropy”

In everyday language, order is often identified with the opposite of entropy. In physics, this equivalence is only partially true. A system can have low entropy yet fail to exhibit a particularly interesting structure; conversely, a system with relatively high entropy can display collective patterns, phase separation, clusters or emergent symmetries at a certain scale.

In practice, at least three distinct planes must be distinguished:

This suggests describing order by means of a vector of observables, not with a single universal scalar.

Total order ≈ (O_m, O_o4, O_o6, O_e, ...)

where each channel captures a different property: modularity, orientation, exclusion, local periodicity, and so forth.

4. Construction of a spectral index on a graph

To make the idea of global order operational, a spectral construction was adopted. A set of particles at positions x_i is represented as a weighted graph, where vertices are the particles and weights describe mutual proximity.

4.1 Weight matrix

A natural choice is a Gaussian kernel:

W_ij = exp[- ||x_i - x_j||² / (2σ²)] , with W_ii = 0

The parameter σ sets the interaction scale. If it is too small, the graph becomes almost disconnected; if too large, local differences are smoothed out. In other words, σ controls the lens through which order is examined.

4.2 Degree matrix and Laplacian

Defining the diagonal degree matrix

D_ii = Σ_j W_ij

the combinatorial Laplacian is built as

L = D - W

or the normalized Laplacian

L_n = I - D^(-1/2) W D^(-1/2)

This second form is particularly useful when local density is not uniform, because it reduces the influence of the mere number of neighbours.

4.3 Spectrum of the Laplacian

The eigenvalues of L_n are ordered as

0 = λ_1 ≤ λ_2 ≤ λ_3 ≤ ... ≤ λ_N

For a connected graph, λ_1 = 0 and is unique. The subsequent eigenvalues contain information about the global geometry of the system. Small values indicate “soft” modes, i.e., slow variations on a large scale; larger values correspond to finer or more irregular structures.

4.4 Spectral energy of low modes

If a configuration possesses a well-defined collective structure, a significant fraction of its “geometric information” is concentrated in a few low modes. This leads to defining spectral concentration indicators, for example:

C_k = (Σ_{m=2}^{k+1} a_m²) / (Σ_{m=2}^{N} a_m²)

where a_m represents the contribution of mode m to a reconstructed field or an equivalent measure. If C_k is high for small k, the configuration is well described by few collective modes.

4.5 Eigengap and structural separation

A very useful index is the spectral gap

gap_k = λ_{k+1} - λ_k

If a marked gap exists after the first low modes, it signals that the system possesses a global structure distinguishable from fine noise. Intuitively, the first modes describe the “skeleton” of the configuration, while the later ones represent secondary details.

5. Physical meaning of Laplacian eigenvalues

The spectral reading is not a purely mathematical artifice. There is a deep connection with real physical phenomena.

5.1 Diffusion and relaxation

The Laplacian governs processes of diffusion, relaxation and mixing on a graph. In dynamics of the type

dφ/dt = -L φ

the eigenmodes decay as

φ_m(t) ∝ exp(-λ_m t)

Small eigenvalues correspond to the slowest decaying modes: they are therefore the most persistent collective components of the structure. For this reason, low modes are natural candidates for encoding global order.

5.2 Clusters and near-disconnections

If the system contains well-separated clusters, the corresponding graph has weak bottlenecks between dense regions. This translates into additional small eigenvalues and into eigenvectors that naturally discriminate the almost-separated components. In other words, clustering leaves a direct signature in the spectrum.

5.3 Coherence and hidden order

A configuration may appear visually irregular yet possess a low number of dominant collective degrees of freedom. In that case, order is not immediately apparent to the eye, but emerges from the spectral behaviour. This is where the idea of hidden order takes shape.

6. From spectral order to a multi-channel taxonomy

Numerical tests quickly showed that a single index, while useful, is not sufficient. The spectral channel captures clusters, modularity and collective structure well, but alone it does not distinguish all types of order. From this arose the idea of an operational taxonomy.

SymbolNameInterpretationType of order detected
Ommodular / spectral ordercoherence of low modes, gap, global structureclusters, domains, modularity
Oo44-fold orientational orderalignment with square symmetryorthogonal lattices, 90° domains
Oo66-fold orientational orderalignment with hexagonal symmetrytriangular / hexagonal lattices
Oeexclusion orderregularity of minimum distance and short-range anti-correlationhard-core, Poisson disk, local repulsion

In this perspective, order is not a single axis but a description space. A configuration can have high values in one channel and low in another. For example:

O_tot is not a universal scalar; it is a profile: O_tot = [ O_m , O_o4 , O_o6 , O_e , ... ]

7. Orientational order: 4-fold and 6-fold channels

To identify square or hexagonal symmetries, the natural observable is the local orientational order parameter.

7.1 Local definition

Given a set of neighbours of particle i, and denoting by θ_ij the bond angle toward neighbour j, one defines:

ψ_k(i) = (1 / n_i) Σ_{j in neighbours(i)} exp( i k θ_ij )

where k = 4 for square symmetry and k = 6 for hexagonal symmetry. The modulus |ψ_k(i)| measures the degree of local alignment with that symmetry.

7.2 Global average

A global index can be obtained as

O_ok = (1/N) Σ_i |ψ_k(i)|

or, in a stricter form, using the complex average before taking the modulus, if global phase coherence is required rather than just local order.

7.3 Interpretation

If the system is an almost perfect square lattice, O_o4 tends to high values; if hexagonal, O_o6 emerges. If the system is random, both channels remain low, apart from small finite-size fluctuations.

7.4 Effect of jitter

Introducing progressive spatial jitter, bond angles disperse and the moduli of the order parameters decrease. This behaviour is valuable because it provides a quantitative response to the degradation of order, not merely a qualitative label.

8. Exclusion order and minimum spacing

There exists a form of regularity that coincides neither with clusters nor with orientational periodicity: simple short-range exclusion. A classic example is the Poisson disk process, in which particles cannot approach each other beyond a certain minimum distance.

This property can produce configurations that are visually “more uniform” than pure randomness, but without generating a true lattice. For this reason, it is useful to introduce a dedicated channel, O_e.

8.1 Measurement idea

One possibility is to consider the nearest-neighbour distances r_i and evaluate their relative dispersion:

μ_r = (1/N) Σ_i r_i σ_r² = (1/N) Σ_i (r_i - μ_r)² CV_r = σ_r / μ_r

A system with more regular minimum spacing has a lower coefficient of variation CV_r. From this one can build, for example, a normalized index of the type

O_e = 1 - normalize(CV_r)

where normalization is calibrated against reference cases.

8.2 Relationship with the pair correlation function

In terms of statistical physics, exclusion order is linked to the short-range behaviour of the radial distribution function g(r). In a system with strong repulsion, g(r) is suppressed near r = 0 and exhibits a sharp rise around a characteristic distance.

This type of order is real and physically relevant, but it coincides neither with crystal symmetry nor with the modularity of a clustered system. Once again: there exist different kinds of order.

9. Reading the results on the 6 configuration families

Extensive tests on 6 families, generated in batches with many independent realizations, clearly showed that different channels respond to different features.

FamilyExpected signaturePhysical reading
Uniform randomlow values in almost all channelsabsence of strong correlations and emergent symmetry
Gaussian blobsOm highstrong modularity and clustering
Hex clusteredOo6 and Om moderate/highlocal hexagonal order and collective organization
Square clusteredOo4 and Om moderate/highsquare symmetry and organized domains
Poisson diskOe high, others modestlocal repulsion without true lattice
Density gradientweak or mixed signalsdensity heterogeneity, not necessarily structural order

This result is important because it avoids a frequent conceptual mistake: considering all “non-random” configurations as equally ordered. In reality:

The taxonomy thus allows a finer and far more faithful reading of the physics of the problem.

10. Connection with statistical physics, correlations and transitions

10.1 Radial distribution function

In the physics of particle systems, one of the standard observables is the radial distribution function g(r), which measures how likely it is to find a particle at distance r from another compared to the random case. An ideal fluid behaves very differently from a system with exclusion or a crystal.

The proposed taxonomy does not replace g(r), but complements it. In particular:

10.2 Structure factor

In the Fourier domain, the classical analogue is the structure factor:

S(k) = (1/N) | Σ_j exp(i k · x_j) |²

Sharp peaks in S(k) indicate periodicity or quasi-periodicity. Here too, the connection with spectral analysis is natural: both methods seek collective coherence, but from different angles. The Laplacian works on connectivity/proximity, the structure factor works on spatial components in reciprocal space.

10.3 Phase transitions and order parameters

In phase transitions, an order parameter grows or decays as a control parameter varies: temperature, density, noise, interaction, disorder. The proposed channels can be interpreted precisely in this spirit. If jitter is increased, or if the interaction potential between particles is modified, one expects a systematic variation of O_o4, O_o6, O_e and O_m.

This opens an interesting perspective: the taxonomy of order could become not only descriptive, but also diagnostic of structural or morphological transitions.

11. A possible multi-scale mathematical synthesis

Because order can exist at different scales, a more robust formulation consists of computing the indices over multiple scale values σ, or over multiple neighbourhood radii, and then combining them.

O_m^(multi) = Σ_s w_s O_m(σ_s) O_o4^(multi) = Σ_s w_s O_o4(σ_s) O_o6^(multi) = Σ_s w_s O_o6(σ_s) O_e^(multi) = Σ_s w_s O_e(σ_s)

where w_s are non-negative weights summing to unity. This avoids entrusting the diagnosis to a single observation scale.

An overall synthesis can then be left in vector form, or compressed into a final score only for practical purposes:

Final score = α O_m + β O_o4 + γ O_o6 + δ O_e

with coefficients α, β, γ, δ chosen according to the application context. Conceptually, however, the vector form remains the most honest, because it preserves the identity of the different types of order.

12. Theoretical limits and future developments

The obtained framework is promising, but should not be presented as a definitive theory. There are many points to be investigated further.

Another interesting aspect concerns the use in fields different from classical particle physics: spatial patterns in images, porous materials, biological aggregates, sensor networks, emergent structures in numerical simulations and even texture analysis.

13. Conclusion

The work session led to a strong and, in some respects, very fruitful conclusion:

Central conclusion: order does not appear as a single quantity, but as a family of phenomena. Entropy measures the multiplicity or uncertainty of possible states; the taxonomy of order distinguishes instead how those possible states are organized in space and in correlations.

In particular, the obtained results suggest that:

The most concise phrase to summarize the meaning of the work might be:

Entropy tells how many ways a system has to be. The taxonomy of order tells in what way those ways are organized.

If this framework is further refined, it could become a useful basis not only for interpreting particle systems, but also for building more general tools for the analysis of complex structures.