Ontologies in physics—9: Derivation of Schrodinger’s equation: context, and essential steps

Updates (corrections, additions, revisions) have been made by 2019.10.28 10:52 IST. Only one of them is explicitly noted, but the others are still there (too many to note separately). However, the essentials of the basic points are kept as they were.

1. The cavity-radiation spectrum:

The continuous spectral-intensity curve for the cavity radiation was established empirically.

Now, before you jump to the Rayleigh-Jeans efforts, or the counting of the EM normal modes in the abstract space, as modern (esp. American) textbooks are wont to do, please take a moment to note an opinion of mine.

1.1 “Cavity-radiation” as a far more informative term:

I believe that we must first come to appreciate the late 19-th century applied physicists (mostly in Germany) who rightly picked up the cavity radiation as the right phenomenon for understanding the matter-light interactions.

Their motivation in studying the cavity radiation was to have a good datum in theory, a good theoretical standard, so that more efficient incandescent light bulbs could be produced for better profits by private businesses. This motivation ultimately paved the way for the discovery of quantum mechanics.

As to the nomenclature of the object/phenomenon they standardized, in my opinion, the term “cavity radiation” is far more informative than the term typically used in modern textbooks, viz. “black-body radiation”. Two reasons:

(i) Only a negligibly small hole on the surface of the cavity acts as a black-body; the rest of the cavity surface does not. Any other approximate realization of the perfectly black-body using a solid object alone, is not as good a choice, because the spectrum a solid body produces depends on the material of the solid (cf. Kirchhoff). The cavity spectrum, however, is independent of the wall-material; its spectrum is dominated by the effects due to pure aether in the cavity than those by the solid wall-material.

(ii) The “cavity-ness” of the body helps in theoretical analysis too, because unlike a structure-less solid continuum, the cavity has very easily demarkable regions for the matter and the eather, i.e., a region each for the material electric oscillators and the light-field. Since the spatial regions occupied by each participating phenomenon is generally different, their roles can be idealized away easily. This is exactly how, in theory, we take away the mass of a mechanical spring as also the stresses in a finite ball, and reach the idealized mechanical system of a massless-spring attached to a point-mass.

1.2 The problem for the theory:

The late 19th-century physicists did a lot of good experimental studies and arrived at the continuous spectrum of the cavity radiation.

The question now was how to explain this spectrum on the basis of the two most fundamental theories known at the time, viz., classical electrodynamics (relevant, because light was known to be EM waves), and thermodynamics including statistical mechanics (relevant, because the cavity was kept heated uniformly to the same temperature, so as to ensure thermal equilibrium between the light field and the cavity walls).

When classical electromagnetic theory was applied to this cavity radiation problem, it could not reproduce the empirically observed curve.

The theory, by Rayleigh and Jean, led to the unrealistic prediction of the ultraviolet catastrophe. It predicted that as you go towards on the higher frequencies side on the graph of the spectrum, the power being emitted by a given frequency (or the infinitesimally small band of frequencies around it) would go on increasing without any upper bound.

Physically speaking, the thermal energy used in keeping the cavity walls heated would be drained by the radiation field in such a fantastic way that the absolute temperature of the internal surface of the cavity wall would have to approach zero. (Notice, the cavity wall is a part of the system here; the environment contains the heat source but not the wall.) No amount of heat supplied at the system surface would be enough to fill the hunger of the aether to convert more and more of the thermal energy of the wall into radiation within itself. Mind you, this circumstance was being proposed by an analysis that was based on a thermodynamic equilibrium. Given a finite supply of heat from the surroundings to the system, the total increase in the internal energy of the system would be finite during any time interval. But since all thermal energy of the wall is converted into light of ever high (even infinitely high) frequencies, none would be left at the internal surface of the wall. In short, a finite wall would develop an infinite temperature gradient at the internal boundary surface.

This was the background when Planck, an accomplished thermodynamicist, picked up this problem for his attention.

2. Planck’s hypothesis:

Note again, in analysis, cavity walls are regarded as being at a thermodynamic equilibrium with the surroundings (so that they maintain a constant and uniform temperature on the entirety of the system boundary just outside of the wall), and also with the light field contained within the cavity (so that an analysis of the electrical oscillators inside the metallic wall of the cavity can provide a clue for the unexpected spectrum of the light).

That’s why, the starting point of Planck’s theorization was not the light field itself but the electrical oscillators in the metal of the wall. The analysis would be conducted for the solid metal, even if, eventually, predictions would be made only for the light field.

On the basis of statistical mechanics, and some pretty abstract “curve-fitting” of sorts [he was not just a skilled “Data Scientist”; he was a gifted one], Planck found that if the energy of the material oscillators were to be, somehow, quantized using the relation:

E = h\nu = \hbar \omega,

then the resulting energy distribution over the various frequencies would be identical to what was observed experimentally. Here, h is the constant Planck used for his abstract “curve-fitting”, \nu is the frequency of the cavity light, \hbar = \dfrac{h}{2\pi} is the modified Planck constant (aka the “reduced” Planck constant because its value is smaller than h), and \omega = 2\pi\nu is the angular frequency of the cavity light.

Ontologically, the significant fact to be noted here is that this analysis deals with the material oscillators, but ends up making an assertion—a quantitative prediction—about the nature of light. The analysis can be justified because the wall and the aether are assumed to be in a thermodynamic equilibrium.

Planck’s original formula was: E = (n+1/2) h \nu where n = 1, 2, 3, \dots. However, in the interest of simplicity (of isolating the relevant ontological issues), we have for now set n = 1 and ignored the 1/2 constant.

Homework: Try to relate the ignored quantities with phenomena such as quantum superposition, zero-temperature energy, etc. Hint: Don’t worry about many particles, whether distinguishable or indistinguishable, or phenomena like entanglement specific to many-particle systems.

3. Photoelectric effect: Einstein’s relation for a monochromatic light:

3.1 The Einstein relation:

On the basis of Planck’s energy-quantization hypothesis, which was eventually regarded as governing the light phenomenon itself (and not just the energies of the electrical oscillators inside the cavity wall), Einstein derived the relation:

\vec{p} = \hbar \vec{k}

where \vec{p} is the momentum associated with a monochromatic light wave, and \vec{k} = \dfrac{2\pi}{\lambda}\hat{e} is the wavevector, \lambda is the wavelength, and \hat{e} is the unit vector in the direction in which the wave travels.

How did Einstein arrive at this relation?

3.2 Einstein postulates particles of light to explain photo-electric effect:

As seen above, what Planck had postulated (ca 1900) was the quantization of energy for the cavity wall oscillators; hence for the wall-to-light field energy exchange; hence for the light-field itself. Yet, Planck did not propose particles of light in place of the continuous field of light in the cavity.

It was Einstein who postulated (ca. 1905) that the spatially continuous field of light be replaced by hypothetical, spatially discrete, particles of light. (It was G. N. Lewis, the then Dean of Chemistry at Berkeley, who, 21 years later, in 1926, coined the term “photon” for them.)

Einstein thought that a particulate nature of light was necessary in order to explain the existence of discrete steps in the phenomenon he studied, viz., the photo-electric effect. [This is not true; the photo-electric effect involves not just light per say, but energy transfers between light and matter; see our comment near the end of this section.]

Einstein then arrived at an expression for the momentum of a photon.

3.2 Momentum of the photon using the light particle postulate:

According to the theory of special relativity (i.e. the classical EM of Maxwell and Lorentz reformulated by Poincare et al. and published ca. 1905 also by Einstein), the energy for a free (unforced) relativistic massive particle is given by the so-called “E = mc^2” equation that even hippies know about; see, for instance, here [^] for clarification of the mass term involved in it:

E = mc^2


E^2 = (m\,c^2)^2 = (\gamma\,m_0\,c^2)^2 = (p\,c)^2 + (m_0\,c^2)^2,

where E is the relativistic energy of a classical massive particle, m is its relativistic mass, c is the speed of light, \gamma = \dfrac{1}{\sqrt{1-(\dfrac{v}{c})^2}} is the Lorentz factor (which indicates physical phenomena like the Lorentz contraction and time dilation, the Lorentz boost, etc.), m_0 is the rest mass of the particle, and p = \gamma m_0\,v is the relativistic momentum of the particle.

According to Einstein, the theory of special relativity must apply to his light particle just as well as it does to the massive bodies. So, he would have the above-given equation govern his light particle’s dynamics, too. However, realizing that such a particle would have to be massless, Einstein set m_0 = 0 for the light particle. Thus, the above equation became, for his photon,

E^2 = (pc)^2, i.e.,

E = pc.

3.3 Momentum of light waves using the Maxwellian EM:

Actually, the same equation can also be derived assuming the electromagnetic wave nature for light (using Pyonting’s vector etc.). Then the distinctive character of the EM waves highlighted by this relation becomes apparent.

The classical NM-ontological waves (like the transverse waves on strings) do not result in a net transport of momentum, though they do transport energy. That’s because the scalar of energy varies as the square of the wave displacement, but the vector of momentum varies as the displacement vector itself. That’s in the classical NM-ontological waves.

In contrast, the “classical” EM-ontological waves transport momentum too, not just energy. That’s their distinctive feature. See David Morin’s online book on waves for explanations (I think chapter 8).

In short, the relation E = pc is basically mandated by the Maxwell’s theory itself.

The special relativistic relations are just a direct consequence of Maxwell’s theory. (The epistemological scope of the special relativity is identical to that of “classical” EM, not greater.)

All in all, you don’t have to assume a particle of light to have the energy-momentum relation for light, in short.

Einstein, however, took this relation apply to the massless particle of the photon hypothesized by him, as detailed in the preceding discussion.

3.4. Einstein lifts the expression for energy of classical waves, and directly uses it for his particles of light—without any pause or explanation:

Now, from the classical wave theory, \omega = ck for any classical wave, where k = \dfrac{2\,\pi}{\lambda} is the wavevector, and \lambda is the wavelength.

Notice that this relation applies only to the oscillations of the material oscillators in the cavity wall as also to aether-waves, but not to structureless particles of light. Did it bother Einstein? I think not.

Einstein did not put forth any argument to show why or how his light particle would obey the same relation. He gave no explanation for how \omega is to be interpreted in the context of his photons.

The fact of the matter is, if you assume a structure-less photon in an empty space, then you cannot explain how the frequency—a property of waves—can at all be an attribute of a point-particle of a photon. Mark my words: structure-less. Nature performs no local changes over time unless there is an internal structure to a spatially discrete particle. A solid body may have angular momentum, but each infinitesimal point-particle comprising it doesn’t. Something similar, for the photon. Einstein gave no description of the structure of the photon or the physical mechanism why it should carry a frequency attribute. … I should know, because I followed this Einstein-Feynman approach for too long, including during my PhD. The required maths of the wave-vector additions involved in a photon’s propagation through space won’t work unless you presume some internal structure to the photon, some device of keeping track of the net wave-vector by the photon.

Thus, Einstein had \omega = ck for his particles of light too—somehow.

3.5. Einstein reaches the momentum–wavevector relation known by his name:

Einstein then accepted Planck’s quantum hypothesis as being valid for his light-particles too, including the exact relation Planck had for the oscillatory phenomena (including waves):

E = \hbar \omega.

Einstein then substituted the relativistic light particle‘s equation E = pc on the left hand-side of Planck’s hypothesis (even though in Planck’s theory, this E was for oscillations/waves), and the classical wave relation \omega = ck on the right hand-side. Accordingly, he got, for his particle of light:

pc = \hbar ck,

i.e., cancelling out c,

p = \hbar k.

This is called the Einstein relation in QM.

3.6. Einstein as the physicist who introduced the wave-particle duality in physics:

Go through the subsections 3.4 and 3.5 again, and take a moment to realize the nature of what Einstein had done.

Einstein became the first man to put forth the wave-particle duality as an acceptable feature of a theory (and not just a conjecture). In effect, he put forth this duality as an essential feature of physics, because it was introduced at the most fundamental levels of theory. And, he did so without bothering to explain what he meant by it.

I gather that Einstein did not experience hesitation while doing so. (He was, you know Einstein! (That hair! That smile!! That very scientist-ness!!!))

4. Some comments on Einstein’s hypothesis of light particles:

4.1 Waves can explain the photo-electric effect; you don’t need Einstein’s particles to explain it:

To explain the photoelectric effect, it is enough to suppose that the light absorption process occurs in the following way: (1) Light is continuously spread in space as a field (as a “wave”), but its emission or absorption occurs only in spatially discrete regions—these processes occur at atoms. (2) An instance of an absorption process remains ongoing only for a finite period of time, but during this interval, it occurs continuously throughout. (3) The nature of the absorption process is such that it either goes to full completion, or it completely reverses as if no energy exchange had at all occurred.

To anticipate our development in the next post, and to give a caricature of the actual physics involved here: The energy is continuously transferred to an atom from the surrounding field. (Physically, this means that the infinitely spread field of light gets further concentrated at the nucleus of the atom, which serves as the reference point due to the singularity of fields at it.). After the process of the continuous energy transfer gets going, the process, for some reason to be supplied (by solving the measurement problem), snaps to one of the energy eigenvalues in the end (like an electrical switch snaps to either on or off position). With this snapping, the energy transfer process comes to a definite end. Hence the quantum nature of the eigenvalue-to-eigenvalue snapping-out–snapping-in process.

In this way, the absorption process, if it at all goes to completion, still results in only a certain quantum of energy being imparted to the absorber; an arbitrary amount of energy (say one-third of a quantum of energy) cannot be transferred in such a process.

In short, a quantized energy transfer process can still be realized without there being a spatially delimited particle of light travelling in space—as Einstein imagined.

4.2. No one highlighted Einstein’s error of introducing particles for light, because his theory had the same maths:

Notice that Einstein begins with the relativistic equation for a massive particle. In the cavity radiation set-up, this can only mean the electric charges (like the protons and electrons) in the cavity wall.

Even for waves in classical material media (like acoustic waves through air/metal), inertia is only a parameter, and not a variable of the wave dynamics. It co-determines the wave-speed in the medium, but beyond that, it has no other role to play. Inertia does not determine forces being exchanged by the wave phenomenon.

The aether anyway does not show any inertia in any electromagnetic phenomenon. So, Einstein’s assumption of the zero inertial effects in the expression for the energy of the photon is perfectly OK—if at all there is a particulate nature for light.

The existence of the thermodynamic equilibrium between the oscillating material charges in the wall and the waves in the aether implies that out of the total relativistic energy of a massive charge, only the pc part gets exchanged with the aether (in Einstein’s view, with the photon); the m_0\,c^2 part must remain with the massive charged particle in the wall.

The aether in cavity has no other means to acquire energy except as through an exchange with the EC Objects in the wall.

Therefore, the internal energy of the aether increases by a quantity that is numerically equal to the pc component of the energy lost by the massive charge.

Overall, Einstein’s assumption of a spatially discrete particle of light is not at all justified, even though the maths he proposed on that conceptual basis still makes perfect sense—it gives the same expression as that for light waves. See the Nobel laureate W. E. Lamb’s paper “Anti-photon” for fascinating discussion [^].

And, yes, Einstein is the original inventor of the mystical idea of the wave-particle duality.

5. Some remarks on Bohr:

We will skip going into Bohr’s theory of the hydrogen atom, primarily because there are hardly any ontological remarks to be made in reference to this theory other than that it was a very ad-hoc kind of a model—though it did predict to great accuracy some of the most interesting and salient features of the hydrogen atom.

Bohr’s was not an ordinary achievement. What he built was a good theoretical model in place of, and to explain, the mere algebraic correlations of the atomic spectra series as given by those formulae by Balmer, Paschen, et al.

If Bohr’s contribution to QM were to end at his 1913 model, he would have made for an ontologically very uninteresting a figure. Who remembers Jean Perin when it comes to the ontological discussions of the continuum vs. particles-based viewpoints? Perin won a physics Nobel for proving the atomic nature of matter, and yet, no one remembers him, because though Perin did fundamental work, he didn’t raise controversies. The knowledge he created has been silently absorbed in the integrated view of physics. Unlike Bohr and Einstein.

That’s why, the fact that Bohr’s model does not invite too many ontological remarks (other than that it is a very tentative, ad-hoc kind of a model) precisely also is the reason why it is best to ignore Bohr at this stage.

Regardless of the physics issues clarified and raised by his Nobel-winning work, we can’t regard Bohr’s model itself as being irritating—certainly not from an ontological viewpoint.

But Bohr, qua a father figure of the mainstream QM as it happened to get developed, of course is very irritating! All in all, any irritance we experience because of him, must be located in his other thoughts, not in his model of the hydrogen atom.

6. de Broglie’s hypothesis of matter-waves:

6.1. de Broglie postulates matter waves:

Light had long been thought (since the ca. 1801 experiment of interference of light-waves by Young) to have a wave nature—i.e., a spatially continuous phenomenon. So, following Einstein’s hypothesis, what was always a spatially continuous phenomenon now also acquired a spatially discrete character. Light always was waves, but also became particles.

The atomic nature of matter was well-established by now. Einstein was the leading physicists of those who must be credited to have helped this theory gain wide acceptance.

Bohr even had a theory for explaining emission / absorption spectra of the hydrogen atom—a model with spatially discrete nucleus and spatially discrete electrons. So, atoms were not just a hypothesis; they were an established fact. And, all parts of them were spatially discrete and finite in extent too.

Matter was particulate in nature. Discrete clumps of clay etc.

Then, following Einstein’s lead, a young Frenchman by name de Broglie put forth the hypothesis, in his PhD thesis, that what is regarded as particulate matter should also have a wave character. Accordingly, there should be waves of matter.

6.2. de Broglie supplies a physical explanation for the stability of the Bohr atom:

de Broglie went even further, and suggested that a massive particle like the electron in the Bohr atom must obey the same relations as are given by the Planck-Einstein relations for light. [Mark this point well; we will shortly make a comment on it. But to continue in the meanwhile…] He then proceeded to do calculations on this basis.

In Bohr’s theory, stationary orbits for the massive electron had been only postulated; they had not been explained on the basis of any physical principle or explanation that was more fundamental or wider in scope. Bohr’s orbits were stationary—by postulate. And, only Bohr’s orbits were stationary—by postulate.

On the basis of his matter-waves hypothesis, de Broglie could now explain the stability of the Bohr orbits (and of only the Bohr orbits). de Broglie pointed that the orbits being closed circles, the matter-waves associated with an electron must form standing waves on them. But standing waves are possible only for certain values of radii, which means that only certain values of angular momenta or energies were allowed for the electrons.

de Broglie thus became the first physicist to employ the eigenvalue paradigm for the dynamics of the electron in a stable hydrogen atom.

6.3. de Broglie’s limitations—mathematical, and ontological:

However, as the later theory of Schrodinger would show (which came within a year and a half), de Broglie’s analysis was too simple. de Broglie was wrong on two counts:

(i) The transverse matter-waves, according to de Broglie, existed with reference to a 1D curve (the Bohr circle) embedded in the 3D space as the reference neutral axis. He couldn’t think of filling the entire 3D space with his matter waves—which is what Schrodinger eventually did.

(ii) de Broglie also altered the ontological character of electrons from massive point-particles to the unexplained “hybrid” or “composite” of: massive point-particles and matter-waves.

From the ontological viewpoint, thus, de Broglie is the originator of the wave-particle duality for the massive particles of electrons, just the way Einstein was the originator of the wave-particle duality for the massless particles of light.

Einstein, of course, beat de Broglie by some 19 years in proposing any such a duality in the first place. (That hair! That smile!! That very scientist-ness!!!)

6.4. What no one notices about de Broglie’s relations:

de Broglie’s relations are nothing but the same old Planck- and Einstein-relations, but now seen as being applied to matter waves, not light. Thus the same equations

E = \hbar \omega and

p = \hbar k

are now known as de Broglie’s relations.

Notice the curious twist here: The equations de Broglie proposed for matter waves were actually derived for the massless phenomenon of light. The m_0-containing term was set to zero in deriving them.

I do not know if any one raised any objections on this basis or not, and whether or how de Broglie answered those objections. However, this issue sure is ontologically interesting. I will leave pursuing the questions it raises as a homework for you, at least for now. [I may cover it in a later post, if required.]

Homework: Why should the equations of a massless phenomenon (viz. the light) apply to the waves of matter?

(Hint: Look at our far too prolonged discussions and seemingly endless repetition of the fact that cavity radiation analysis applies at thermal equilibrium. Also refer to our preference for the term “cavity radiation,” and not “black-body radiation.” … That should be enough of a hint.)

…As a side remark: The quantum theory anyway had begun to get developed at a furious pace by 1924, and a relativistic theory for quantum mechanics would be given by Dirac just a few years later. Relativistic QM is out of the scope of our present series of posts.

7. Before getting into the derivation of Schrodinger’s equation:

7.1. The place of Schrodinger’s equation in the quantum theory:

We now come to the equation that has held sway over physicists’ imagination for almost a century (95 years, to be precise), viz., the linear partial differential equation (PDE) that was inductively derived by Schrodinger.

[Ignore Feynman here when he says that Schrodinger’s procedure is not a derivation. It is a derivation, but it is an inductive derivation, not a deductive one. Feynman artificially constrained the concept of derivation only to deduction. Expected of him.]

In terms of the commanding position of Schrodinger’s equation, every valid implication of QM, every non-intuitive feature of it, every interpretational issue about it, every debate in the QM history,… they all trace themselves back to some or the other term in this equation, or some or the other aspect of it, or some or the other fact assumed or implied by its analysis scheme—its overall nature.

If you have a nagging issue about QM, and if you can’t trace it back to Schrodinger’s equation (at least with a form of it, as in the relativistic QM), then the issue, we could even say, does not exist! All the empirical evidence we have so far points in that direction.

Either your issue is there explicitly in Schrodinger’s equation, or at least implicitly in its context, or in one of its concrete or abstract implications. Or, the issue simply isn’t there—physically.

Including the worst riddle of QM, viz., the measurement problem. This problem is a riddle precisely because of a mathematical nature of Schrodinger’s equation, viz., that it’s a linear PDE.

So, we want to highlight this fact:

Even if all that you want to do is to “just” solve the measurement problem, you still have to work with the Schrodinger equation—including its inductive context, the ontology it presupposes (at least implicitly), its mathematical structure and form, and all their implications.

The reason is: there is only one primary-unknown variable in the Schrodinger equation, viz., \Psi(x,t). And, there is only one more field, viz., V(x,t) in it. The rest are either constants or the space- and time-variables over which the fields are defined.

There is no place for any additional variable in QM—known or unknown. The reason for this, in turn, is: Schrodinger’s equation predicts all the known QM phenomena with astounding accuracy. That’s why, there is no place for hidden variables either—the very idea itself is plain wrong.You don’t have to make an appeal to a detail like Bell’s theorem. The mathematical nature of Schrodinger’s equation, and the predictive success, together say that.

Therefore, solving the measurement problem must “only” require some ontological, physical, or mathematical reorganization involving the same old \Psi(x,t) and V(x,t) variables, and the same old constants (not to mention the same x and t variables over which the two fields are defined).

That’s why it is important to develop a good intuition about each term in this equation, about how the terms are put together, etc. To start developing such an intuition (which we will formalize in the ontology of Schrodinger’s QM in the next post), it is necessary to look into the logical scheme of its inductive derivation.

Without any loss of the essential physical meaning, (and perhaps with a greater clarity about physical meaning), we will use only the energy-based analysis in the derivation here, not the full-fledged variationally-based analysis which Schrodinger had originally performed in deriving his equation (by appeal to an analogy of mechanics with geometrical optics). We will more or less directly follow David Morin’s presentation. (It’s the best in the “town” for the purposes of a learner.)

7.1. Energy analysis with single numbers (or the aspatial, system-level, variables):

In energy analysis, the total energy content of an isolated mechanical system of objects that exert only conservative forces on each other, can be given as a sum of the kinetic and potential energies.

E = T + \Pi

where E denotes the total internal energy number of the system (here, not the magnitude of the electric force field), T is the kinetic energy number, and \Pi is the potential energy number.

Notice that as stated just as above, and without any further addition to the equation, this is not a statement of energy conservation principle; it is a statement that the internal energy for an isolated system having conservative forces consists of two and only two forms: kinetic and potential. (We ignore the heat, for instance.)

Speaking properly, a statement for energy conservation here would have been:

\oint \text{d}E = 0 = \oint \text{d}T + \oint \text{d}\Pi.

That is, a cyclic change for the total energy number for an isolated system is zero, and therefore, the sum of cyclic changes in its kinetic and potential energy numbers also must be zero—assuming that potentials are produced by conservative forces (as the electrostatic forces are). Now, for conservative forces, \oint \text{d}\Pi turns out to be zero, and so, the cyclic change in the kinetic energy too must be zero. Thus:

\oint \text{d}E = 0 = \oint \text{d}\Pi =  \oint \text{d}T.

However, notice, for non-cyclic changes, the most informative statement would be a differential equation; it would read:

\text{d}E = 0 = \text{d}T + \text{d}\Pi

This is because \text{d}E = 0 for any change in an isolated system. However, in general, notice that:

\text{d}T = - \text{d}\Pi \neq 0.

By integration between any two arbitrary states, we get

E = T + \Pi = \text{a constant},

which says that the E number stays the same for an isolated system—it is conserved—in any arbitrary change. Notice that this is a statement of energy conservation—due to the addition of the last equality. The addition of the last equality looks trivial, but it is in fact necessary to be noted explicity. We will work with this form of the equation.

Both T and \Pi are still aspatial numbers here. Since we have thrashed out this topic thoroughly in the previous posts of this series, we will not go into the distinction of the aspatial variables vs. the spatially defined quantities/fields once again. We will simply proceed to bring the aspatial variables down from their Platonic Lagrangian “heaven” to our analysis formulated in reference to the physical space (which is, in practice, 3D).

7.2. Energy analysis for the mechanics of a massive point-particle moving on a curve:

Now, in classical mechanics, for a massive-point particle,

T(x,t) = \dfrac{1}{2}mv(x,t)^2 = \dfrac{1}{2}\dfrac{p(x,t)^2}{2m}.


E = \dfrac{p(x,t)^2}{2m} + \Pi(x,t)

In particle mechanics, p(x,t)^2 = \vec{p}(x,t)\cdot\vec{p}(x,t) always lies with the instantaneous position x(t) of the massive particle. In the above equation, \Pi(x,t) should still remain an aspatial variable, but it’s common practice in the Variational/Lagrangian/Hamiltonian mechanics to assume that this function is specified without any particular reference to particle position, and therefore,

\Pi(x,t) not is only a known quantity, it also is independent of p(x,t).

Under this scope-narrowing assumption, it is OK to think of a 1D field for the potential energy function (e.g. a curved wire over which the bead slides under gravity with the time-dependent geometry of the wire not depending on the position or the kinetic energy function of the bead).

Accordingly the potential energy number \Pi(x,t) can be represented via a 1D field of V(x,t).

Thus we have:

E = \dfrac{p(x,t)^2}{2m} + V(x,t),

where  V(x,t) and p(x,t) are not functions of each other. Note,

In classical mechanics of point-particles (and their interactions with fields), though V(x,t) is a field, p(x,t) still remains a point-property at the particle’s position. So, E(x,t) may also be taken to be a point-property of the particle.

This may look like hair-splitting to most modern physicists. However, it is not. The reason we went to such great lengths in identifying the conditions under which an energy can be regarded as an aspatial attribute of the system, and the conditions under which it can be regarded to have an identifiable existence in space—whether at the position of a point-particle or all over the domain as a field—are matters having crucial bearing on the kind of ontology there is assumed for the objects in the system.

In Schrodinger’s equation, it eventually turns out that V(x,t) is assumed to be a field, and not only that, but, effectively, also is the momentum function p(x,t). In developing Schrodinger’s equation, we also have to be careful not to directly assign the aspatial variable E to successive points of space—thus, hold on before you convert E to E(x,t). The reason is, Schrodinger’s equation deals with fields, not particles. The equation E = T +\Pi applies equally well to systems of particles as well as to systems of particles and fields, and to systems of only fields. Be careful. (I am correcting some of my own slightly misleading statements below.)

8. Specific steps comprising the essential scheme of Schrodinger’s derivation:

8.1. There should be a wave PDE for de Broglie’s matter-waves:

Schrodinger became intrigued by de Broglie’s theory, and within months, gave a seminar on it at his university. Debye (of the Debye-Scherrer camera fame, among other achievements) was in attendance, and casually remarked:

if the electron is a matter wave, then there must be a wave equation for it.

Debye actually meant a partial differential equation when he said “a wave equation.” A wave equation relates some spatio-temporal changes in the wave variable. The V is a field (following our EM ontology; see previous posts in this series), and the wave variable also must be a 3D field.

8.2. The wave ansatz:

The simplest ansatz to assume for a wave function (i.e. a field) in the 3D physical space, is the plane-wave:

\Psi(x,t) = A e^{i(kx - \omega t)}

With the negative sign put only on \omega but not on k, we get a (co)sinusoidal wave that travels to the right (i.e., in the direction of the positive x-axis). We will consider only the plane-wave traveling in the x-direction, for simplicity; however, realize, in a 3D physical space, two more plane-waves, one each in y– and z-directions, will be required.

8.3. The energy conservation equation that reflects the de Broglie relations:

We need to somehow relate k and \omega to the the Planck-Einstein relations as used in de Broglie’s theory (i.e. as applying to the massive electron). To do so, take the specialized energy conservation statement explained above, viz.

E = \dfrac{p(x,t)^2}{2m} + V(x,t)

which is an equation for a particle at x(t), with its momentum also located at its position.

We then substitute de Broglie’s relations for matter waves in it—i.e., we use the mathematical equation of Planck’s for E on the left hand-side, and Einstein’s for p on the right hand-side. We thus get:

\hbar \omega = \dfrac{\hbar^2 k^2}{2m} + V(x,t).

Notice, we have brought in the wave-particle duality, implicit in the de Broglie relations, now into an equation which in classical mechanics was only for particles.

Update: As an after-thought, a better way to look at is to begin with the aspatial-variables equation:

E = \dfrac{p^2}{2m} + \Pi = \text{a constant},

then make an electrostatic field for \Pi by substituting V(x,t) in its place, so as to arrive at:

E = \dfrac{p^2}{2m} + V(x,t) = \text{a constant},

and then, without worrying about whether E, T or p are defined in the physical space or not, to take this equation as applying to the system as a whole, and proceed to the next step. Accordingly, I am also slightly modifying the discussion below.

8.4. Making the energy conservation equation with the de Broglie terms, refer to the wave ansatz:

To relate the above equation to the plane-wave ansatz, multiply all the terms by \Psi(x,t), and get:

\hbar \omega \Psi(x,t) = \dfrac{\hbar^2 k^2}{2m}\Psi(x,t) + V(x,t)\Psi(x,t) = \text{a constant}\Psi(x,t).

The preceding step might look a bit quizzical, but doing so comes in handy soon enough to keep the mathematics sensible.

Physically, what the step does is to convert the first or the total term (total energy, which is conserved) from the aspatial variable E (or a system-attribute of \text{a constant}) to a spatially distributed entity—because of the multiplication by \Psi(x,t), a field. \Psi basically distributes the system-wide global variable (an aspatial variable) to all points in the physical space. BTW, this is the basic physical reason why \Psi(x,t) has to be normalized—we don’t want to change the value of the conserved quantity of the total energy.

Similarly, the same step also converts the second term (the kinetic energy, now expressed using the momentum) from the aspatial variable p^2/2m to a spatially distributed field.

The \Psi(x,t) variable refers to matter waves in 1D, but can be easily generalized to 3D—unlike de Broglie’s standing waves on the circles of the Bohr orbits.

Why do we not have to offer a physical mechanism for the multiplications by \Psi(x,t)? The answer is: it is its absence which is ontologically impossible to interpret. What comes as physically existing in the physical 3D space are the fields, and \Psi(x,t) helps in pinning their quantities. It is the aspatial variables/numbers that are devices of calculations, not the fields.

8.5. Transforming the energy conservation equation having the de Broglie terms into a partial differential equation:

Now, to get to the wave equation, we note the partial differentiations of the wave ansatz:

\dfrac{\partial \Psi}{\partial x} = ik\,A e^{i(kx - \omega t)} = ik\,\Psi(x,t),

and so,

\dfrac{\partial^2 \Psi}{\partial x^2} = -k^2\,\Psi

which implies that

k^2\Psi = - \dfrac{1}{\Psi}\dfrac{\partial^2\Psi}{\partial x^2}.

On the time-side, the first-order differential is enough, if eliminating \omega is our concern:

\dfrac{\partial \Psi}{\partial t} = -i\,\omega\,A e^{i(kx - \omega t)} = -i\,\omega\,\Psi

which implies that

\omega\Psi(x,t) = \dfrac{1}{-i}\dfrac{\partial \Psi}{\partial t} = i \dfrac{\partial \Psi}{\partial t}

Now, simple: Plug and chug! The energy conservation equation with the de Broglie terms goes from:

\hbar \omega \Psi(x,t) = \dfrac{\hbar^2 k^2}{2m}\Psi(x,t) + V(x,t)\Psi(x,t).


i\,\hbar \dfrac{\partial \Psi(x,t)}{\partial t} =\ -\, \dfrac{\hbar^2}{2m}\dfrac{\partial^2\Psi(x,t)}{\partial x^2} + V(x,t)\Psi(x,t)

That’s the most general (time-dependent) Schrodinger equation for you!

8.6. A few comments:

1. There is the imaginary root of unity, viz. i on the left hand-side. So, the general solution must be complex-valued.

2. The PDE obtained has the space-derivative to the second order, but the time-derivative only to the first order.

In classical waves (i.e. the NM-ontological waves like the waves on strings, as well as in EM-waves), the both the space- and time-derivatives are to the second order. That’s because the classical waves are real-valued. If you have complex-valued waves, then the first order derivative is enough to get oscillations in time. Complex-valued waves are mandated because we inserted de Broglie’s relations in the energy conservation equation. In classical NM-ontological mechanics, we would have kept the \Psi real-valued, and so, would have to take its second-order time differential. But then, none of the energy terms in the energy conservation equation would obey Planck’s hypothesis, hence Einstein’s relation, and hence, de Broglie’s relations. In short, the complex-valued nature of \Psi is mandated, ultimately, by Planck’s hypothesis and a wave ansatz.

3. For later reference, note that:

The kinetic energy is a field, given by:

T(x,t) =\ -\,\dfrac{\hbar^2}{2m} \dfrac{1}{\Psi}\dfrac{\partial^2\Psi}{\partial x^2},

which suggests the following definition for the momentum in the system:

\vec{p}(x,t) =\ i\,\hbar \dfrac{1}{\Psi(x,t)} \nabla \Psi(x,t).

Thus, both kinetic energy and momentum are fields in the Schrodinger equation. The mainstream view regards them as fields defined on the abstract, 3ND configuration space.

In contrast, we take all fields of the Schrodinger equation: V(x,t), \Psi(x,t), and hence, T(x,t) as well as \vec{p}(x,t) as the 3D fields in aether. (The last two become fields because their terms involve \Psi).

The ontological and mathematical justification for our view that they are fields in the 3D physical space should be too simple and obvious by now (at this stage in this series of posts). The only thing to look into, now, is to justify that \Psi(x,t) field remains a 3D field even when there are two or more particles. We touch upon this issue in the next post, when we come to the ontology of QM.

9. Some comments on the development of Schrodinger’s equation, from our ontological viewpoint:

Notice very carefully the funny circling around going on here, with respect to light and matter, waves and particles, massless aether and massive objects. (We touched upon many of these points above and before, so they will get repeated, unfortunately! However, it’s important to put them together in one place for easy reference later on.)

Both Planck and Einstein began with the energy analysis of massive charged objects (roughly, the EC Objects of our EM ontology). They then ascribed the quantities of E = \hbar \omega and p = \hbar k to the massless aether. This procedure is justifiable because of the equality in the energy or momentum exchanges at the thermodynamic equilibrium.

As to the ontology, Planck had his own doubts about the “transfer” of quantization in the energy states of oscillators to the quantization of the EM fields. He regarded the quantization of energy of the radiation field as only a hypothesis. With the enormous benefit of hindsight (of more than a century), we can say that his hesitation for quantizing energy fields themselves was not justified. Schrodinger showed how they could be continuous (and continuously changing) entities and still obey the energy-eigenvalue equations for stationary states.

In contrast to Planck, Einstein was daring, actually brazen. He didn’t have any issue with quantization. In fact, he went much beyond, actually overboard in our opinion, and introduced also the spatial quantization to light, by introducing particles of light.

Notice, the sizes of attributes, i.e. the magnitudes of energy (or momentum), involved in the exchanges between the material point-charges and the aether is the same. However,

The fact that an exchange of energy is possible (that the sizes of the respective attributes can undergo changes in a mutually compensating way) does not alter the very ontological nature of the respective objects which enter into the interactions.

Ontologically, an EC object still remains a point-particle (more on this in the next post), and the aether still remains a spatially spread-out, non-mechanical, object—even when they interact, and therefore, even when their abstract measures like energies can change, and these changes be equated.

In our view, energy and momentum are point-properties when possessed by point-particles of EC Objects; they should be seen as moving in space with these objects (more on it, in the next post); they never exist at any other locations. In contrast, energy and momentum are field properties when possessed by the aether; they remain spread over all space at all times; they never concentrate in one place.

To equate the sizes of attributes is not to change the ontological character.

We do not blur the point-particle into a smear over space, neither do we collapse a field to a point. We simply say that from a more abstract, thermodynamic-systems perspective, the quantities of attributes called energy and momentum, in case of both types of objects, come to have the same magnitudes under equilibrium exchanges, that’s all.

In converting some quantity of steel into a piece of gold or vice versa, the respective physical objects remain the same; they only change hands, that’s all. The quantity of steel does not become golden, nor can a coin of gold be used in building a car, just because they got exchanged.

Einstein however confused this ontological issue and prescribed an exchange of not just quantities but also of the basic ontological characters. He put forth the idea of spatially discrete particles of light (later called photons).

de Broglie then entered the scene, compounded Einstein’s ontological error on the other hand of the ontological division, and prescribed an ontological wave character to the matter particles. He in effect smeared out matter into space, and also made the smear dance everywhere as a wave—a “symmetrical” counterpart to how Einstein was the original “inventor” of “anti-smearing” fields in space to a point-object, and then, of making this point-object (of the photon) go everywhere while a carrying wave attribute with it—but without any explanation about the internal structure which might lead to its having the wave attribute.

In effect, Einstein and de Broglie were the initiators of the wave-particle duality. Both their works implied, in the absence of any satisfactory explanation coming forth from either of them for their ontological transgressions, the riddles: The riddle of how the wave field “collapses” to the wave-attribute of the photon in Einstein’s theory, and how the mass and charge smear out in de Broglie’s theory.

Both must have been influence by bad elements of philosophy, including ontology. But the Copenhagen camp went further, much further.

The logical-positivistically minded Bohr, Heisenberg et al. of the Copenhagen camp then seized the moment, and formalized the measurement problem via the wavefunction “collapse,” the Complementarity Principle, etc. etc. etc. And despite all the “celebrated” debates, neither Einstein nor de Broglie ever realized that, as far as physics was concerned, it was they who had set the ball rolling in the first place!

Schrodinger didn’t think of questioning these ontological transgressions—neither did any one else. He merely improved the maths of it—by generalizing the eigenvalue problem from the original de Broglie waves on 1D curves to a similar problem for his wavefunction \Psi, initially in the 3D space. Then, in the absence of sufficient clarity regarding the nature of Lagrangian abstractions (and their nature to the physical 3D space), Schrodinger even took \Psi (following Lorentz’s objection) to the abstract 3ND configuration space.

A quarter of a century later, John von Neumann used his formidable skills in mathematical abstractions, and, as might be expected, also equipped with a perfect carelessness about ontology, took all the QM-related confusions, and cast them all into the concrete, by situating the entire theoretical structure QM on the “floating grounds” of an infinite-dimensional Hilbert space, with \Psi and V of course “living” in abstract 3ND configuration space.

Oh, BTW, regardless of his otherwise well-earned reputation, there were errors in von Neumann’s proofs too. It took decades before a non-mainstream non-American QM physicist, named John S. Bell, discovered an important one. Bell said:

The proof of von Neumann is not merely false but foolish!” [^].

I am tempted to ask Bell:

“Why just von Neumann, John? Weren’t they all at least partly both?”

The only way to counter all their errors is to clearly understand all the aspects of all such issues—by and for yourself. You must understand the epistemology and ontology involved in the issues (yes, this one, first!), also physics (both “classical” and QM), and then, also the relation of mathematics to physics to ontology and epistemology in general. But once you do that, you find that all their silly errors and objections have evaporated away.

10. Operators are not ontologically important:

I do not know who began to emphasize operators in QM. Dirac? von Neumann? Still others?

But the notion has become entrenched in the mainstream QM. A lot of store is set up on the idea that the classical variables must be represented, in theory, by operators—by objects that are, in Feynman’s memorable words “hungry” forever. The operators for the momentum and energy, for instance, are respectively given as:

\hat{p} = i\,\hbar \nabla


\hat{H} =\ -\ \dfrac{\hbar^2}{2m} \dfrac{\partial^2}{\partial x^2} + V(x,t).

To somehow have everything fit their operator-primary theory, they also carefully formulated the notion that the operator of a number, a variable, or a field function, when it “acts” on \Psi, results in just plain multiplication of that mathematical object with the \Psi—without any explanation or justification on the physical grounds. Why multiply when Nature does no multiplications—without there being a mechanism acting to that effect? Blank out.

It all is more than just a bit weird, but it’s there—the operator-primacy theory of formulating QM. And I am sure that it has some carefully crafted and elegant-looking mathematical basis intelligently created for it too—complete with carefully noted notations, definitions, lemmas, theorems, proofs, etc. All forming such a hugely abstract and obfuscating structure that errors in proofs are kept well hidden for decades.

For now, just note that the very notion of operators itself is not very important when it comes to ontological discussions. Naturally, the finer distinctions about it like the linear operators, Hermitian operators, etc., also is not at all important. Just my personal opinion. But it’s been reached with a good ontological understanding of the issues, I think.

11. A preview of the things to come next:

OK. In this post, we touched on many of the finer points having ontological implications. The next time, we will provide our answer regarding the proper ontology of QM.

We will refer to the physics of only the simplest quantum system, viz. the hydrogen atom (and comparable quantum models, notably, the particle in the box (PIB), the quantum harmonic oscillator, and similar 1-particle quantum systems). We will make a formal list all the objects used in the QM ontology, and also indicate the kind of 3D aetherial field there has to be, for the system wavefunction. We will also discuss some analogies that help understand the nature of the \Psi field. For instance, we will point out the fact that \Psi exists even in the PIB model, i.e., even at places in the domain where V is zero. There are some interesting repercussions arising out of this fact.

We will also touch upon the fact that the action-at-a-distance is absent in our EM ontology, and hence, it should also be absent in our QM ontology. However, the presence of the direct-action does not mean that there cannot be changes that occur simultaneously everywhere in the aether. The two phenomena are slightly different, and we will delineate them. The non-relativistic QM theory, in particular, requires the latter.

We will, however, not touch upon the measurement problem. Understanding the measurement problem requires understanding two new physics topics to a certain depth and with sufficient scope: QM of many-particle systems, and the physics of the nonlinear differential equations. Both are vast topics in themselves. Further, tackling the measurement problem doesn’t change the list of different ontological objects that are involved in QM or their basic nature. In fact, measurement problem is rather a specifically detailed physics problem. Solving it, IMHO, does require a very good clarity on the QM ontology, but the basic ontological scheme remains the same as for the single hydrogen atom. That’s why, we won’t be touching on that topic. Experts in QM may refer to the Outline document I have already put out, earlier this year, at iMechanica [^]. All the rest: Well, you have wait, or ask the experts, what else?

One particular aspect of the many-particle quantum systems which is very much in vogue these days is the entanglement. Since we won’t be covering the many-particle systems in this series, we also wouldn’t be touching on the physics of quantum mechanical entanglement. However, at least as of today, I do not very clearly see if the phenomenon of entanglement requires us to make any substantial changes in the ontology of QM. In fact, I think not. Entanglement complicates only the physics of QM, but not its ontology. Hence the planned omission of entanglement from this series.

So, all in all, our description of the QM ontology itself would get completed right in the next post. And, with it, also this ontological series would come to an end.

Of course, my blogging would continue, as usual. So, I might write occasional posts on these topics: many-particle QM systems, nonlinearity proposed by me in the Schrodinger equation, the measurement problem, the quantum entanglement, and then, perhaps, also the quantum spin. However, there won’t be a continuously executed project of a series of posts as such, to cover these topics. I will simply write on these topics on a more or less “random” and occasional basis—whenever I feel like.

So there. Check out the next—i.e. the last—post in this series, when it comes, say in a week’s time or so. In the meanwhile, go through the previous posts if you have joined late.

Also, have a happy Diwali!

Alright, take care, and bye for now…

A song I like:

(Hindi) “dheere dheere machal ae dil-e-beqarar”
Music: Hemant Kumar
Singer: Lata Mangeshkar
Lyrics: Kaifi Aazmi
[Credits listed in a random order]


— First published: 2019.10.26 18:37 IST
— Corrected typos, added sub-section headings, revised some contents (without touching the points), added a few explanations, etc., by 2019.10.27 11:28 IST. Will now this post (~7,500 words!) as is. At least until this series gets over.
Update on 2019.10.28 10:50 IST: Still corrected some misleading passages, added notes for better clarification, corrected typos, etc. (~8,825 words!). Let’s leave it. I need to really turn to writing the next post.