| This paper aims to propose a new compact and economical neural model capable of containing a massive amount of information. The model is based on level-dependent informational functions, in which different information functions are applied at different levels. By increasing the number of levels and corresponding functions, it is expected that a larger amount of information can be processed by relatively smaller networks. We propose the simplest model as a first approximation, in which the weights are assumed to be organized into two distinct levels, namely, collective and conditional. By maximizing differential potentiality, defined as the difference between collective and conditional potentiality, more information is expected to be generated. In addition, differential potentiality is applied bidirectionally, meaning that it can be both maximized and minimized simultaneously in order to unify the two types of informational functions. The method was applied to the classification of French texts into verse and prose. By using mutual information to represent structural properties in long-distance correlations, mutual information was found to be higher for verse, whereas conditional entropy was higher for prose. In addition, we observed that, during training, weights farther from the base point became increasingly important for verse-prose classification. The results suggest that verse, which imposes strong constraints on ordinary natural language, attempts to make full use of long-distance relations. Although such relations tend to weaken as the distance between the target character and the base character increases, verse maintains a relatively high level of mutual information, thereby preserving structural organization. This indicates that a compact model incorporating long-distance relations can store and process more information with a minimal number of components, ultimately leading to a compact and economical neural architecture. |
*** Title, author list and abstract as submitted during Camera-Ready version delivery. Small changes that may have occurred during processing by Springer may not appear in this window.