# Determinism, Indeterminism, Probability, and the nature of the laws of physics—a second take…

After I wrote the last post [^], several points struck me. Some of the points that were mostly implicit needed to be addressed systematically. So, I began writing a small document containing these after-thoughts, focusing more on the structural side of the argument.

However, I don’t find time to convert these points + statements into a proper write-up. At the same time, I want to get done with this topic, at least for now, so that I can better focus on some other tasks related to data science. So, let me share the write-up in whatever form it is in, currently. Sorry for its uneven tone and all (compared to even my other writing, that is!)

Causality as a concept is very poorly understood by present-day physicists. They typically understand only one sense of the term: evolution in time. But causality is a far broader concept. Here I agree with Ayn Rand / Leonard Peikoff (OPAR). See the Ayn Rand Lexicon entry, here [^]. (However, I wrote the points below without re-reading it, and instead, relying on whatever understanding I have already come to develop starting from my studies of the same material.)

Physical universe consists of objects. Objects have identity. Identity is the sum total of all characteristics, attributes, properties, etc., of an object. Objects act in accordance with their identity; they cannot act otherwise. Interactions are not primary; they do not come into being without there being objects that undergo the interactions. Objects do not change their respective identities when they take actions—not even during interactions with other objects. The law of causality is a higher-level view taken of this fact.

In the cause-effect relationship, the cause refers to the nature (identity) of an object, and the effect refers to an action that the object takes (or undergoes). Both refer to one and the same object. TBD: Trace the example of one moving billiard ball undergoing a perfectly elastic collision with another billiard ball. Bring out how the interaction—here, the pair of the contact forces—is a name for each ball undergoing an action in accordance with its nature. An interaction is a pair of actions.

A physical law as a mapping (e.g., a function, or even a functional) from inputs to outputs.

The quantitative laws of physics often use the real number system, i.e., quantification with infinite precision. An infinite precision is a mathematical concept, not physical. (Expect physicists to eternally keep on confusing between the two kinds of concepts.)

Application of a physical law traces the same conceptual linkages as are involved in the formulation of law, but in the reverse direction.

In both formulation of a physical law and in its application, there is always some regime of applicability which is at least implicitly understood for both inputs and outputs. A pertinent idea here is: range of variations. A further idea is the response of the output to small variations in the input.

Example: Prediction by software whether a cricket ball would have hit the stumps or not, in an LBW situation.

The input position being used by the software in a certain LBW decision could be off from reality by millimeters, or at least, by a fraction of a millimeter. Still, the law (the mapping) is such that it produces predictions that are within small limits, so that it can be relied on.

Two input values, each theoretically infinitely precise, but differing by a small magnitude from each other, may be taken to define an interval or zone of input variations. As to the zone of the corresponding output, it may be thought of as an oval produced in the plane of the stumps, using the deterministic method used in making predictions.

The nature of the law governing the motion of the ball (even after factoring in aspects like effects of interaction with air and turbulence, etc.) itself is such that the size of the O/P zone remains small enough. (It does not grow exponentially.) Hence, we can use the software confidently.

That is to say, the software can be confidently used for predicting—-i.e., determining—the zone of possible landing of the ball in the plane of the stumps.

Overall, here are three elements that must be noted: (i) Each of the input positions lying at the extreme ends of the input zone of variations itself does have an infinite precision. (ii) Further, the mapping (the law) has theoretically infinite precision. (iii) Each of the outputs lying at extreme ends of the output zone also itself has theoretically infinite precision.

Existence of such infinite precision is a given. But it is not at all the relevant issue.

What matters in applications is something more than these three. It is the fact that applications always involve zones of variations in the inputs and outputs.

Such zones are then used in error estimates. (Also for engineering control purposes, say as in automation or robotic applications.) But the fact that quantities being fed to the program as inputs themselves may be in error is not the crux of the issue. If you focus too much on errors, you will simply get into an infinite regress of error bounds for error bounds for error bounds…

Focus, instead, on the infinity of precision of the three kinds mentioned above, and focus on the fact that in addition to those infinitely precise quantities, application procedure does involve having zones of possible variations in the input, and it also involves the problem estimating how large the corresponding zone of variations in the output is—whether it is sufficiently small for the law and a particular application procedure or situation.

In physics, such details of application procedures are kept merely understood. They are hardly, if ever, mentioned and discussed explicitly. Physicists again show their poor epistemology. They discuss such things in terms not of the zones but of “error” bounds. This already inserts the wedge of dichotomy: infinitely precise laws vs. errors in applications. This dichotomy is entirely uncalled for. But, physicists simply aren’t that smart, that’s all.

“Indeterministic mapping,” for the above example (LBW decisions) would the one in which the ball can be mapped as going anywhere over, and perhaps even beyond, the stadium.

Such a law and the application method (including the software) would be useless as an aid in the LBW decisions.

However, phenomenologically, the very dynamics of the cricket ball’s motion itself is simple enough that it leads to a causal law whose nature is such that for a small variation in the input conditions (a small input variations zone), the predicted zone of the O/P also is small enough. It is for this reason that we say that predictions are possible in this situation. That is to say, this is not an indeterministic situation or law.

Not all physical situations are exactly like the example of the predicting the motion of the cricket ball. There are physical situations which show a certain common—and confusing—characteristic.

They involve interactions that are deterministic when occurring between two (or few) bodies. Thus, the laws governing a simple interaction between one or two bodies are deterministic—in the above sense of the term (i.e., in terms of infinite precision for mapping, and an existence of the zones of variations in the inputs and outputs).

But these physical situations also involve: (i) a nonlinear mapping, (ii) a sufficiently large number of interacting bodies, and further, (iii) coupling of all the interactions.

It is these physical situations which produce such an overall system behaviour that it can produce an exponentially diverging output zone even for a small zone of input variations.

So, a small change in I/P is sufficient to produce a huge change in O/P.

However, note the confusing part. Even if the system behaviour for a large number of bodies does show an exponential increase in the output zone, the mapping itself is such that when it is applied to only one pair of bodies in isolation of all the others, then the output zone does remain non-exponential.

It is this characteristic which tricks people into forming two camps that go on arguing eternally. One side says that it is deterministic (making reference to a single-pair interaction), the other side says it is indeterministic (making reference to a large number of interactions, based on the same law).

The fallacy arises out of confusing a characteristic of the application method or model (variations in input and output zones) with the precision of the law or the mapping.

Example: N-body problem.

Example: NS equations as capturing a continuum description (a nonlinear one) of a very large number of bodies.

Example: Several other physical laws entering the coupled description, apart from the NS equations, in the bubbles collapse problem.

Example: Quantum mechanics

The Law vs. the System distinction: What is indeterministic is not a law governing a simple interaction taken abstractly (in which context the law was formed), but the behaviour of the system. A law (a governing equation) can be deterministic, but still, the system behavior can become indeterministic.

Even indeterministic models or system designs, when they are described using a different kind of maths (the one which is formulated at a higher level of abstractions, and, relying on the limiting values of relative frequencies i.e. probabilities), still do show causality.

Yes, probability is a notion which itself is based on causality—after all, it uses limiting values for the relative frequencies. The ability to use the limiting processes squarely rests on there being some definite features which, by being definite, do help reveal the existence of the identity. If such features (enduring, causal) were not to be part of the identity of the objects that are abstractly seen to act probabilistically, then no application of a limiting process would be possible, and so not even a definition probability or randomness would be possible.

The notion of probability is more fundamental than that of randomness. Randomness is an abstract notion that idealizes the notion of absence of every form of order. … You can use the axioms of probability even when sequences are known to be not random, can’t you? Also, hierarchically, order comes before does randomness. Randomness is defined as the absence of (all applicable forms of) orderliness; orderliness is not defined as absence of randomness—it is defined via the some but any principle, in reference to various more concrete instances that show some or the other definable form of order.

But expect not just physicists but also mathematicians, computer scientists, and philosophers, to eternally keep on confusing the issues involved here, too. They all are dumb.

Summary:

Let me now mention a few important take-aways (though some new points not discussed above also crept in, sorry!):

• Physical laws are always causal.
• Physical laws often use the infinite precision of the real number system, and hence, they do show the mathematical character of infinite precision.
• The solution paradigm used in physics requires specifying some input numbers and calculating the corresponding output numbers. If the physical law is based on real number system, than all the numbers used too are supposed to have infinite precision.
• Applications always involve a consideration of the zone of variations in the input conditions and the corresponding zone of variations in the output predictions. The relation between the sizes of the two zones is determined by the nature of the physical law itself. If for a small variation in the input zone the law predicts a sufficiently small output zone, people call the law itself deterministic.
• Complex systems are not always composed from parts that are in themselves complex. Complex systems can be built by arranging essentially very simpler parts that are put together in complex configurations.
• Each of the simpler part may be governed by a deterministic law. However, when the input-output zones are considered for the complex system taken as a whole, the system behaviour may show exponential increase in the size of the output zone. In such a case, the system must be described as indeterministic.
• Indeterministic systems still are based on causal laws. Hence, with appropriate methods and abstractions (including mathematical ones), they can be made to reveal the underlying causality. One useful theory is that of probability. The theory turns the supposed disadvantage (a large number of interacting bodies) on its head, and uses limiting values of relative frequencies, i.e., probability. The probability theory itself is based on causality, and so are indeterministic systems.
• Systems may be deterministic or indeterministic, and in the latter case, they may be described using the maths of probability theory. Physical laws are always causal. However, if they have to be described using the terms of determinism or indeterminism, then we will have to say that they are always deterministic. After all, if the physical laws showed exponentially large output zone even when simpler systems were considered, they could not be formulated or regarded as laws.

In conclusion: Physical laws are always causal. They may also always be regarded as being deterministic. However, if systems are complex, then even if the laws governing their simpler parts were all deterministic, the system behavior itself may turn out to be indeterministic. Some indeterministic systems can be well described using the theory of probability. The theory of probability itself is based on the idea of causality albeit measures defined over large number of instances are taken, thereby exploiting the fact that there are far too many objects interacting in a complex manner.

A song I like:

(Hindi) “ho re ghungaroo kaa bole…”
Singer: Lata Mangeshkar
Music: R. D. Burman
Lyrics: Anand Bakshi

/

# The self-field, and the objectivity of the classical electrostatic potentials: my analysis

This blog post continues from my last post, and has become overdue by now. I had promised to give my answers to the questions raised last time. Without attempting to explain too much, let me jot down the answers.

1. The rule of omitting the self-field:

This rule arises in electrostatic interactions basically because the Coulombic field has a spherical symmetry. The same rule would also work out in any field that has a spherical symmetry—not just the inverse-separation fields, and not necessarily only the singular potentials, though Coulombic potentials do show both these latter properties too.

It is helpful here to think in terms of not potentials but of forces.

Draw any arbitrary curve. Then, hold one end of the curve fixed at the origin, and sweep the curve through all possible angles around it, to get a 3D field. This 3D field has a spherical symmetry, too. Hence, gradients at the same radial distance on opposite sides of the origin are always equal and opposite.

Now you know that the negative gradient of potential gives you a force. Since for any spherical potential the gradients are equal and opposite, they cancel out. So, the forces cancel out to.

Realize here that in calculating the force exerted by a potential field on a point-particle (say an electron), the force cannot be calculated in reference to just one point. The very definition of the gradient refers to two different points in space, even if they be only infinitesimally separated apart. So, the proper procedure is to start with a small sphere centered around the given electron, calculate the gradients of the potential field at all points on the surface of this sphere, calculate the sum of the forces exerted on the domain contained inside the spherical surface by these forces, and then take the sphere to the limiting of vanishing size. The sum of the forces thus exerted is the net force acting on that point-particle.

In case of the Coulombic potentials, the forces thus calculated on the surface of any sphere (centered on that particle) turn out to be zero. This fact holds true for spheres of all radii. It is true that gradients (and forces) progressively increase as the size of the sphere decreases—in fact they increase without all bounds for singular potentials. However, the aforementioned cancellation holds true at any stage in the limiting process. Hence, it holds true for the entirety of the self-field.

In calculating motions of a given electron, what matters is not whether its self-field exists or not, but whether it exerts a net force on the same electron or not. The self-field does exist (at least in the sense explained later below) and in that sense, yes, it does keep exerting forces at all times, also on the same electron. However, due to the spherical symmetry, the net force that the field exerts on the same electron turns out to be zero.

In short:

Even if you were to include the self-field in the calculations, if the field is spherically symmetric, then the final net force experienced by the same electron would still have no part coming from its own self-field. Hence, to economize calculations without sacrificing exactitude in any way, we discard it out of considerations.The rule of omitting the self-field is just a matter of economizing calculations; it is not a fundamental law characterizing what field may be objectively said to exist. If the potential field due to other charges exists, then, in the same sense, the self-field too exists. It’s just that for the motions of the self field-generating electron, it is as good as non-existent.

However, the question of whether a potential field physically exists or not, turns out to be more subtle than what might be thought.

2. Conditions for the objective existence of electrostatic potentials:

It once again helps to think of forces first, and only then of potentials.

Consider two electrons in an otherwise empty spatial region of an isolated system. Suppose the first electron ($e_1$), is at a position $x_1$, and a second electron $e_2$ is at a position $x_2$. What Coulomb’s law now says is that the two electrons mutually exert equal and opposite forces on each other. The magnitudes of these forces are proportional to the inverse-square of the distance which separates the two. For the like charges, the forces is repulsive, and for unlike charges, it is attractive. The amount of the electrostatic forces thus exerted do not depend on mass; they depend only the amounts of the respective charges.

The potential energy of the system for this particular configuration is given by (i) arbitrarily assigning a zero potential to infinite separation between the two charges, and (ii) imagining as if both the charges have been brought from infinity to their respective current positions.

It is important to realize that the potential energy for a particular configuration of two electrons does not form a field. It is merely a single number.

However, it is possible to imagine that one of the charges (say $e_1$) is held fixed at a point, say at $\vec{r}_1$, and the other charge is successively taken, in any order, at every other point $\vec{r}_2$ in the infinite domain. A single number is thus generated for each pair of $(\vec{r}_1, \vec{r}_2)$. Thus, we can obtain a mapping from the set of positions for the two charges, to a set of the potential energy numbers. This second set can be regarded as forming a field—in the $3D$ space.

However, notice that thus defined, the potential energy field is only a device of calculations. It necessarily refers to a second charge—the one which is imagined to be at one point in the domain at a time, with the procedure covering the entire domain. The energy field cannot be regarded as a property of the first charge alone.

Now, if the potential energy field $U$ thus obtained is normalized by dividing it with the electric charge of the second charge, then we get the potential energy for a unit test-charge. Another name for the potential energy obtained when a unit test-charge is used for the second charge is: the electrostatic potential (denoted as $V$).

But still, in classical mechanics, the potential field also is only a device of calculations; it does not exist as a property of the first charge, because the potential energy itself does not exist as a property of that fixed charge alone. What does exist is the physical effect that there are those potential energy numbers for those specific configurations of the fixed charge and the test charge.

This is the reason why the potential energy field, and therefore the electrostatic potential of a single charge in an otherwise empty space does not exist. Mathematically, it is regarded as zero (though it could have been assigned any other arbitrary, constant value.)

Potentials arise only out of interaction of two charges. In classical mechanics, the charges are point-particles. Point-particles exist only at definite locations and nowhere else. Therefore, their interaction also must be seen as happening only at the locations where they do exist, and nowhere else.

If that is so, then in what sense can we at all say that potential energy (or electrostaic potential) field does physically exist?

Consider a single electron in an isolated system, again. Assume that its position remains fixed.

Suppose there were something else in the isolated system—-something—some object—every part of which undergoes an electrostatic interaction with the fixed (first) electron. If this second object were to be spread all over the domain, and if every part of it were able to interact with the fixed charge, then we could say that the potential energy field exists objectively—as an attribute of this second object. Ditto, for the electric potential field.

Note three crucially important points, now.

2.1. The second object is not the usual classical object.

You cannot regard the second (spread-out) object as a mere classical charge distribution. The reason is this.

If the second object were to be actually a classical object, then any given part of it would have to electrostatically interact with every other part of itself too. You couldn’t possibly say that a volume element in this second object interacts only with the “external” electron. But if the second object were also to be self-interacting, then what would come to exist would not be the simple inverse-distance potential field energy, in reference to that single “external” electron. The space would be filled with a very weird field. Admitting motion to the property of the local charge in the second object, every locally present charge would soon redistribute itself back “to” infinity (if it is negative), or it all would collapse into the origin (if the charge on the second object were to be positive, because the fixed electron’s field is singular). But if we allow no charge redistributions, and the second field were to be classical (i.e. capable of self-interacting), then the field of the second object would have to have singularities everywhere. Very weird. That’s why:

If you want to regard the potential field as objectively existing, you have to also posit (i.e. postulate) that the second object itself is not classical in nature.

Classical electrostatics, if it has to regard a potential field as objectively (i.e. physically) existing, must therefore come to postulate a non-classical background object!

2.2. Assuming you do posit such a (non-classical) second object (one which becomes “just” a background object), then what happens when you introduce a second electron into the system?

You would run into another seeming contradiction. You would find that this second electron has no job left to do, as far as interacting with the first (fixed) electron is concerned.

If the potential field exists objectively, then the second electron would have to just passively register the pre-existing potential in its vicinity (because it is the second object which is doing all the electrostatic interactions—all the mutual forcings—with the first electron). So, the second electron would do nothing of consequence with respect to the first electron. It would just become a receptacle for registering the force being exchanged by the background object in its local neighborhood.

But the seeming contradiction here is that as far as the first electron is concerned, it does feel the potential set up by the second electron! It may be seen to do so once again via the mediation of the background object.

Therefore, both electrons have to be simultaneously regarded as being active and passive with respect to each other. They are active as agents that establish their own potential fields, together with an interaction with the background object. But they also become passive in the sense that they are mere point-masses that only feel the potential field in the background object and experience forces (accelerations) accordingly.

The paradox is thus resolved by having each electron set up a field as a result of an interaction with the background object—but have no interaction with the other electron at all.

2.3. Note carefully what agency is assigned to what object.

The potential field has a singularity at the position of that charge which produces it. But the potential field itself is created either by the second charge (by imagining it to be present at various places), or by a non-classical background object (which, in a way, is nothing but an objectification of the potential field-calculation procedure).

Thus, there arises a duality of a kind—a double-agent nature, so to speak. The potential energy is calculated for the second charge (the one that is passive), in the sense that the potential energy is relevant for calculating the motion of the second charge. That’s because the self-field cancels out for all motions of the first charge. However,

The potential energy is calculated for the second charge. But the field so calculated has been set up by the first (fixed) charge. Charges do not interact with each other; they interact only with the background object.

2.4. If the charges do not interact with each other, and if they interact only with the background object, then it is worth considering this question:

Can’t the charges be seen as mere conditions—points of singularities—in the background object?

Indeed, this seems to be the most reasonable approach to take. In other words,

All effects due to point charges can be regarded as field conditions within the background object. Thus, paradoxically enough, a non-classical distributed field comes to represent the classical, massive and charged point-particles themselves. (The mass becomes just a parameter of the interactions of singularities within a $3D$ field.) The charges (like electrons) do not exist as classical massive particles, not even in the classical electrostatics.

3. A partly analogous situation: The stress-strain fields:

If the above situation seems too paradoxical, it might be helpful to think of the stress-strain fields in solids.

Consider a horizontally lying thin plate of steel with two rigid rods welded to it at two different points. Suppose horizontal forces of mutually opposite directions are applied through the rods (either compressive or tensile). As you know, as a consequence, stress-strain fields get set up in the plate.

From an external viewpoint, the two rods are regarded as interacting with each other (exchanging forces with each other) via the medium of the plate. However, in reality, they are interacting only with the object that is the plate. The direct interaction, thus, is only between a rod and the plate. A rod is forced, it interacts with the plate, the plate sets up stress-strain field everywhere, the local stress-field near the second rod interacts with it, and the second rod registers a force—which balances out the force applied at its end. Conversely, the force applied at the second rod also can be seen as getting transmitted to the first rod via the stress-strain field in the plate material.

There is no contradiction in this description, because we attribute the stress-strain field to the plate itself, and always treat this stress-strain field as if it came into existence due to both the rods acting simultaneously.

In particular, we do not try to isolate a single-rod attribute out of the stress-strain field, the way we try to ascribe a potential to the first charge alone.

Come to think of it, if we have only one rod and if we apply force to it, no stress-strain field would result (i.e. neglecting inertia effects of the steel plate). Instead, the plate would simply move in the rigid body mode. Now, in solid mechanics, we never try to visualize a stress-strain field associated with a single rod alone.

It is a fallacy of our thinking that when it comes to electrostatics, we try to ascribe the potential to the first charge, and altogether neglect the abstract procedure of placing the test charge at various locations, or the postulate of positing a non-classical background object which carries that potential.

In the interest of completeness, it must be noted that the stress-strain fields are tensor fields (they are based on the gradients of vector fields), whereas the electrostatic force-field is a vector field (it is based on the gradient of the scalar potential field). A more relevant analogy for the electrostatic field, therefore might the forces exchanged by two point-vortices existing in an ideal fluid.

4. But why bother with it all?

The reason I went into all this discussion is because all these issues become important in the context of quantum mechanics. Even in quantum mechanics, when you have two charges that are interacting with each other, you do run into these same issues, because the Schrodinger equation does have a potential energy term in it. Consider the following situation.

If an electrostatic potential is regarded as being set up by a single charge (as is done by the proton in the nucleus of the hydrogen atom), but if it is also to be regarded as an actually existing and spread out entity (as a $3D$ field, the way Schrodinger’s equation assumes it to be), then a question arises: What is the role of the second charge (e.g., that of the electron in an hydrogen atom)? What happens when the second charge (the electron) is represented quantum mechanically? In particular:

What happens to the potential field if it represents the potential energy of the second charge, but the second charge itself is now being represented only via the complex-valued wavefunction?

And worse: What happens when there are two electrons, and both interacting with each other via electrostatic repulsions, and both are required to be represented quantum mechanically—as in the case of the electrons in an helium atom?

Can a charge be regarded as having a potential field as well as a wavefunction field? If so, what happens to the point-specific repulsions as are mandated by the Coulomb law? How precisely is the $V(\vec{r}_1, \vec{r}_2)$ term to be interpreted?

I was thinking about these things when these issues occurred to me: the issue of the self-field, and the question of the physical vs. merely mathematical existence of the potential fields of two or more quantum-mechanically interacting charges.

Guess I am inching towards my full answers. Guess I have reached my answers, but I need to have them verified with some physicists.

5. The help I want:

As a part of my answer-finding exercises (to be finished by this month-end), I might be contacting a second set of physicists soon enough. The issue I want to learn from them is the following:

How exactly do they do computational modeling of the helium atom using the finite difference method (FDM), within the context of the standard (mainstream) quantum mechanics?

That is the question. Once I understand this part, I would be done with the development of my new approach to understanding QM.

I do have some ideas regarding the highlighted question. It’s just that I want to have these ideas confirmed from some physicists before (or along-side) implementing the FDM code. So, I might be approaching someone—possibly you!

Please note my question once again. I don’t want to do perturbation theory. I would also like to avoid the variational method.

Yes, I am very comfortable with the finite element method, which is basically based on the variational calculus. So, given a good (detailed enough) account of the variational method for the He atom, it should be possible to translate it into the FEM terms.

However, ideally, what I would like to do is to implement it as an FDM code.

So there.

Please suggest good references and / or people working on this topic, if you know any. Thanks in advance.

A song I like:

[… Here I thought that there was no song that Salil Chowdhury had composed and I had not listened to. (Well, at least when it comes to his Hindi songs). That’s what I had come to believe, and here trots along this one—and that too, as a part of a collection by someone! … The time-delay between my first listening to this song, and my liking it, was zero. (Or, it was a negative time-delay, if you refer to the instant that the first listening got over). … Also, one of those rare occasions when one is able to say that any linear ordering of the credits could only be random.]

Music: Salil Chowdhury
Lyrics: Gulzaar
Singer: Lata Mangeshkar

/

# The rule of omitting the self-field in calculations—and whether potentials have an objective existence or not

There was an issue concerning the strictly classical, non-relativistic electricity which I was (once again) confronted with, during my continuing preoccupation with quantum mechanics.

Actually, a small part of this issue had occurred to me earlier too, and I had worked through it back then.

However, the overall issue had never occurred to me with as much of scope, generality and force as it did last evening. And I could not immediately resolve it. So, for a while, especially last night, I unexpectedly found myself to have become very confused, even discouraged.

Then, this morning, after a good night’s rest, everything became clear right while sipping my morning cup of tea. Things came together literally within a span of just a few minutes. I want to share the issue and its resolution with you.

The question in question (!) is the following.

Consider 2 (or $N$) number of point-charges, say electrons. Each electron sets up an electrostatic (Coulombic) potential everywhere in space, for the other electrons to “feel”.

As you know, the potential set up by the $i$-th electron is:
$V_i(\vec{r}_i, \vec{r}) = \dfrac{1}{4 \pi \epsilon_0} \dfrac{Q_i}{|\vec{r} - \vec{r}_i|}$
where $\vec{r}_i$ is the position vector of the $i$-th electron, $\vec{r}$ is any arbitrary point in space, and $Q_i$ is the charge of the $i$-th electron.

The potential energy associated with some other ($j$-th) electron being at the position $\vec{r}_j$ (i.e. the energy that the system acquires in bringing the two electrons from $\infty$ to their respective positions some finite distance apart), is then given as:
$U_{ij}(\vec{r}_i, \vec{r}_j) = \dfrac{1}{4 \pi \epsilon_0} \dfrac{Q_i\,Q_j}{|\vec{r}_j - \vec{r}_i|}$

The notation followed here is the following: In $U_{ij}$, the potential field is produced by the $i$-th electron, and the work is done by the $j$-th electron against the $i$-th electron.

Symmetrically, the potential energy for this configuration can also be expressed as:
$U_{ji}(\vec{r}_j, \vec{r}_i) = \dfrac{1}{4 \pi \epsilon_0} \dfrac{Q_j\,Q_i}{|\vec{r}_i - \vec{r}_j|}$

If a system has only two charges, then its total potential energy $U$ can be expressed either as $U_{ji}$ or as $U_{ij}$. Thus,
$U = U_{ji} = U_{ij}$

Similarly, for any pair of charges in an $N$-particle system, too. Therefore, the total energy of an $N$-particle system is given as:
$U = \sum\limits_{i}^{N} \sum\limits_{j = i+1}^{N} U_{ij}$

The issue now is this: Can we say that the total potential energy $U$ has an objective existence in the physical world? Or is it just a device of calculations that we have invented, just a concept from maths that has no meaningful physical counterpart?

(A side remark: Energy may perhaps exist as an attribute or property of something else, and not necessarily as a separate physical object by itself. However, existence as an attribute still is an objective existence.)

The reason to raise this doubt is the following.

When calculating the motion of the $i$-th charge, we consider only the potentials $V_j$ produced by the other charges, not the potential produced by the given charge $V_i$ itself.

Now, if the potential produced by the given charge ($V_i$) also exists at every point in space, then why does it not enter the calculations? How does its physical efficacy get evaporated away? And, symmetrically: The motion of the $j$-th charge occurs as if $V_j$ had physically evaporated away.

The issue generalizes in a straight-forward manner. If there are $N$ number of charges, then for calculating the motion of a given $i$-th charge, the potential fields of all other charges are considered operative. But not its own field.

How can motion become sensitive to only a part of the total potential energy existing at a point even if the other part also exists at the same point? That is the question.

This circumstance seems to indicate as if there is subjectivity built deep into the very fabric of classical mechanics. It is as if the universe just knows what a subject is going to calculate, and accordingly, it just makes the corresponding field mystically go away. The universe—the physical universe—acts as if it were changing in response to what we choose to do in our mind. Mind you, the universe seems to change in response to not just our observations (as in QM), but even as we merely proceed to do calculations. How does that come to happen?… May be the whole physical universe exists only in our imagination?

Got the point?

No, my confusion was not as pathetic as that in the previous paragraph. But I still found myself being confused about how to account for the fact that an electron’s own field does not enter the calculations.

But it was not all. A non-clarity on this issue also meant that there was another confusing issue which also raised its head. This secondary issue arises out of the fact that the Coulombic potential set up by any point-charge is singular in nature (or at least approximately so).

If the electron is a point-particle and if its own potential “is” $\infty$ at its position, then why does it at all get influenced by the finite potential of any other charge? That is the question.

Notice, the second issue is most acute when the potentials in question are singular in nature. But even if you arbitrarily remove the singularity by declaring (say by fiat) a finite size for the electron, thereby making its own field only finitely large (and not infinite), the above-mentioned issue still remains. So long as its own field is finite but much, much larger than the potential of any other charge, the effects due to the other charges should become comparatively less significant, perhaps even negligibly small. Why does this not happen? Why does the rule instead go exactly the other way around, and makes those much smaller effects due to other charges count, but not the self-field of the very electron in question?

While thinking about QM, there was a certain point where this entire gamut of issues became important—whether the potential has an objective existence or not, the rule of omitting the self-field while calculating motions of particles, the singular potential, etc.

The specific issue I was trying to think through was: two interacting particles (e.g. the two electrons in the helium atom). It was while thinking on this problem that this problem occurred to me. And then, it also led me to wonder: what if some intellectual goon in the guise of a physicist comes along, and says that my proposal isn’t valid because there is this element of subjectivity to it? This thought occurred to me with all its force only last night. (Or so I think.) And I could not recall seeing a ready-made answer in a text-book or so. Nor could I figure it out immediately, at night, after a whole day’s work. And as I failed to resolve the anticipated objection, I progressively got more and more confused last night, even discouraged.

However, this morning, it all got resolved in a jiffy.

Would you like to give it a try? Why is it that while calculating the motion of the $i$-th charge, you consider the potentials set up by all the rest of the charges, but not its own potential field? Why this rule? Get this part right, and all the philosophical humbug mentioned earlier just evaporates away too.

I would wait for a couple of days or so before coming back and providing you with the answer I found. May be I will write another post about it.

Update on 2019.03.16 20:14 IST: Corrected the statement concerning the total energy of a two-electron system. Also simplified the further discussion by couching it preferably in terms of potentials rather than energies (as in the first published version), because a Coulombic potential always remains anchored in the given charge—it doesn’t additionally depend on the other charges the way energy does. Modified the notation to reflect the emphasis on the potentials rather than energy.

A song I like:

[What else? [… see the songs section in the last post.]]
(Hindi) “woh dil kahaan se laaoon…”
Singer: Lata Mangeshkar
Music: Ravi
Lyrics: Rajinder Kishen

A bit of a conjecture as to why Ravi’s songs tend to be so hummable, of a certain simplicity, especially, almost always based on a very simple rhythm. My conjecture is that because Ravi grew up in an atmosphere of “bhajan”-singing.

Observe that it is in the very nature of music that it puts your mind into an abstract frame of mind. Observe any singer, especially the non-professional ones (or the ones who are not very highly experienced in controlling their body-language while singing, as happens to singers who participate in college events or talent shows).

When they sing, their eyes seem to roll in a very peculiar manner. It seems random but it isn’t. It’s as if the eyes involuntarily get set in the motions of searching for something definite to be found somewhere, as if the thing to be found would be in the concrete physical space outside, but within a split-second, the eyes again move as if the person has realized that nothing corresponding is to be found in the world out there. That’s why the eyes “roll away.” The same thing goes on repeating, as the singer passes over various words, points of pauses, nuances, or musical phrases.

The involuntary motions of the eyes of the singer provide a window into his experience of music. It’s as if his consciousness was again and again going on registering a sequence of two very fleeting experiences: (i) a search for something in the outside world corresponding to an inner experience felt in the present, and immediately later, (ii) a realization (and therefore the turning away of the eyes from an initially picked up tentative direction) that nothing in the outside world would match what was being searched for.

The experience of music necessarily makes you realize the abstractness of itself. It tends to make you realize that the root-referents of your musical experience lie not in a specific object or phenomenon in the physical world, but in the inner realm, that of your own emotions, judgments, self-reflections, etc.

This nature of music makes it ideally suited to let you turn your attention away from the outside world, and has the capacity or potential to induce a kind of a quiet self-reflection in you.

But the switch from the experience of frustrated searches into the outside world to a quiet self-reflection within oneself is not the only option available here. Music can also induce in you a transitioning from those unfulfilled searches to a frantic kind of an activity: screams, frantic shouting, random gyrations, and what not. In evidence, observe any piece of modern American / Western pop-music.

However, when done right, music can also induce a state of self-reflection, and by evoking certain kind of emotions, it can even lead to a sense of orderliness, peace, serenity. To make this part effective, such a music has to be simple enough, and orderly enough. That’s why devotional music in the refined cultural traditions is, as a rule, of a certain kind of simplicity.

The experience of music isn’t the highest possible spiritual experience. But if done right, it can make your transition from the ordinary experience to a deep, profound spiritual experience easy. And doing it right involves certain orderliness, simplicity in all respects: tune, tone, singing style, rhythm, instrumental sections, transitions between phrases, etc.

If you grow up listening to this kind of a music, your own music in your adult years tends to reflect the same qualities. The simplicity of rhythm. The alluringly simple tunes. The “hummability quotient.” (You don’t want to focus on intricate patterns of melody in devotional music; you want it to be so simple that minimal mental exertion is involved in rendering it, so that your mental energy can quietly transition towards your spiritual quest and experiences.) Etc.

I am not saying that the reason Ravi’s music is so great is because he listened his father sing “bhajan”s. If this were true, there would be tens of thousands of music composers having talents comparable to Ravi’s. But the fact is that Ravi was a genius—a self-taught genius, in fact. (He never received any formal training in music ever.) But what I am saying is that if you do have the musical ability, having this kind of a family environment would leave its mark. Definitely.

Of course, this all was just a conjecture. Check it out and see if it holds or not.

… May be I should convert this “note” in a separate post by itself. Would be easier to keep track of it. … Some other time. … I have to work on QM; after all, exactly only half the month remains now. … Bye for now. …

/

It’s nearing January-end already [^]… I am trying very hard to stay optimistic [^].

BTW, remember: (i) this blog is in copyright, (ii) your feedback is welcome.

A song I like:

(Hindi) “agar main kahoon”
Music: Shankar-Ehsaan-Loy
Lyrics: Javed Akhtar
Singers: Alka Yagnik, Udit Narayan

/

# A list of books for understanding the non-relativistic QM

TL;DR: NFY (Not for you).

In this post, I will list those books which have been actually helpful to me during my self-studies of QM.

But before coming to the list, let me first note down a few points which would be important for engineers who wish to study QM on their own. After all, my blog is regularly visited by engineers too. That’s what the data about the visit patterns to various posts says.

Others (e.g. physicists) may perhaps skip over the note in the next section, and instead jump directly over to the list itself. However, even if the note for engineers is too long, perhaps, physicists should go through it too. If they did, they sure would come to know a bit more about the kind of background from which the engineers come.

# I. A note for engineers who wish to study QM on their own:

The point is this: QM is vast, even if its postulates are just a few. So, it takes a prolonged, sustained effort to learn it.

For the same reason (of vastness), learning QM also involves your having to side-by-side learn an entirely new approach to learning itself. (If you have been a good student of engineering, chances are pretty good that you already have some first-hand idea about this meta-learning thing. But the point is, if you wish to understand QM, you have to put it to use once again afresh!)

In terms of vastness, QM is, in some sense, comparable to this cluster of subjects spanning engineering and physics: engineering thermodynamics, statistical mechanics, kinetics, fluid mechanics, and heat- and mass-transfer.

I.1 Thermodynamics as a science that is hard to get right:

The four laws of thermodynamics (including the zeroth and the third) are easy enough to grasp—I mean, in the simpler settings. But when it comes to this subject (as also for the Newtonian mechanics, i.e., from the particle to the continuum mechanics), God lies not in the postulates but in their applications.

The statement of the first law of thermodynamics remains the same simple one. But complexity begins to creep in as soon as you begin to dig just a little bit deeper with it. Entire categories of new considerations enter the picture, and the meaning of the same postulates gets both enriched and deepened with them. For instance, consider the distinction of the open vs. the closed vs. the isolated systems, and the corresponding changes that have to be made even to the mathematical statements of the law. That’s just for the starters. The complexity keeps increasing: studies of different processes like adiabatic vs. isochoric vs. polytropic vs. isentropic etc., and understanding the nature of these idealizations and their relevance in diverse practical applications such as: steam power (important even today, specifically, in the nuclear power plants), IC engines, jet turbines, refrigeration and air-conditioning, furnaces, boilers, process equipment, etc.; phase transitions, material properties and their variations; empirical charts….

Then there is another point. To really understand thermodynamics well, you have to learn a lot of other subjects too. You have to go further and study some different but complementary sciences like heat and mass transfer, to begin with. And to do that well, you need to study fluid dynamics first. Kinetics is practically important too; think of process engineering and cost of energy. Ideas from statistical mechanics are important from the viewpoint of developing a fundamental understanding. And then, you have to augment all this study with all the empirical studies of the irreversible processes (think: the boiling heat transfer process). It’s only when you study such an entire gamut of topics and subjects that you can truly come to say that you now have some realistic understanding of the subject matter that is thermodynamics.

Developing understanding of the aforementioned vast cluster of subjects (of thermal sciences) is difficult; it requires a sustained effort spanning over years. Mistakes are not only very easily possible; in engineering schools, they are routine. Let me illustrate this point with just one example from thermodynamics.

Consider some point that is somewhat nutty to get right. For instance, consider the fact that no work is done during the free expansion of a gas. If you are such a genius that you could correctly get this point right on your very first reading, then hats off to you. Personally, I could not. Neither do I know of even a single engineer who could. We all had summarily stumbled on some fine points like this.

You see, what happens here is that thermodynamics and statistical mechanics involve entirely different ways of thinking, but they both are being introduced almost at the same time during your UG studies. Therefore, it is easy enough to mix up the some disparate metaphors coming from these two entirely different paradigms.

Coming to the specific example of the free expansion, initially, it is easy enough for you to think that since momentum is being carried by all those gas molecules escaping the chamber during the free expansion process, there must be a leakage of work associated with it. Further, since the molecules were already moving in a random manner, there must be an accompanying leakage of the heat too. Both turn out to be wrong ways of thinking about the process! Intuitions about thermodynamics develop only slowly. You think that you understood what the basic idea of a system and an environment is like, but the example of the free expansion serves to expose the holes in your understanding. And then, it’s not just thermo and stat mech. You have to learn how to separate both from kinetics (and they all, from the two other, closely related, thermal sciences: fluid mechanics, and heat and mass transfer).

But before you can learn to separate out the unique perspectives of these subject matters, you first have to learn their contents! But the way the university education happens, you also get exposed to them more or less simultaneously! (4 years is as nothing in a career that might span over 30 to 40 years.)

Since you are learning a lot many different paradigms at the same time, it is easy enough to naively transfer your fledgling understanding of one aspect of one paradigm (say, that of the particle or statistical mechanics) and naively insert it, in an invalid manner, into another paradigm which you are still just learning to use at roughly the same time (thermodynamics). This is what happens in the case of the free expansion of gases. Or, of throttling. Or, of the difference between the two… It is a rare student who can correctly answer all the questions on this topic, during his oral examination.

Now, here is the ultimate point: Postulates-wise, thermodynamics is independent of the rest of the subjects from the aforementioned cluster of subjects. So, in theory, you should be able to “get” thermodynamics—its postulates, in all their generality—even without ever having learnt these other subjects.

Yet, paradoxically enough, we find that complicated concepts and processes also become easier to understand when they are approached using many different conceptual pathways. A good example here would be the concept of entropy.

When you are a XII standard student (or even during your first couple of years in engineering), you are, more or less, just getting your feet wet with the idea of the differentials. As it so happens, before you run into the concept of entropy, virtually every physics concept was such that it was a ratio of two differentials. For instance, the instantaneous velocity is the ratio of d(displacement) over d(time). But the definition of entropy involves a more creative way of using the calculus: it has a differential (and that too an inexact differential), but only in the numerator. The denominator is a “plain-vanilla” variable. You have already learnt the maths used in dealing with the rates of changes—i.e. the calculus. But that doesn’t mean that you have an already learnt physical imagination with you which would let you handle this kind of a definition—one that involves a ratio of a differential quantity to an ordinary variable. … “Why should only one thing change even as the other thing remains steadfastly constant?” you may wonder. “And if it is anyway going to stay constant, then is it even significant? (Isn’t the derivative of a constant the zero?) So, why not just throw the constant variable out of the consideration?” You see, one major reason you can’t deal with the definition of entropy is simply because you can’t deal with the way its maths comes arranged. Understanding entropy in a purely thermodynamic—i.e. continuum—context can get confusing, to say the least. But then, just throw in a simple insight from Boltzmann’s theory, and suddenly, the bulb gets lit up!

So, paradoxically enough, even if multiple paradigms mean more work and even more possibilities of confusion, in some ways, having multiple approaches also does help.

When a subject is vast, and therefore involves multiple paradigms, people regularly fail to get certain complex ideas right. That happens even to very smart people. For instance, consider Maxwell’s daemon. Not many people could figure out how to deal with it correctly, for such a long time.

…All in all, it is only some time later, when you have already studied all these topics—thermodynamics, kinetics, statistical mechanics, fluid mechanics, heat and mass transfer—that finally things begin to fall in place (if they at all do, at any point of time!). But getting there involves hard effort that goes on for years: it involves learning all these topics individually, and then, also integrating them all together.

In other words, there is no short-cut to understanding thermodynamics. It seems easy enough to think that you’ve understood the 4 laws the first time you ran into them. But the huge gaps in your understanding begin to become apparent only when it comes to applying them to a wide variety of situations.

I.2 QM is vast, and requires multiple passes of studies:

Something similar happens also with QM. It too has relatively few postulates (3 to 6 in number, depending on which author you consult) but a vast scope of applicability. It is easy enough to develop a feeling that you have understood the postulates right. But, exactly as in the case of thermodynamics (or Newtonian mechanics), once again, the God lies not in the postulates but rather in their applications. And in case of QM, you have to hasten to add: the God also lies in the very meaning of these postulates—not just their applications. QM carries a one-two punch.

Similar to the case of thermodynamics and the related cluster of subjects, it is not possible to “get” QM in the first go. If you think you did, chances are that you have a superhuman intelligence. Or, far, far more likely, the plain fact of the matter is that you simply didn’t get the subject matter right—not in its full generality. (Which is what typically happens to the CS guys who think that they have mastered QM, even if the only “QM” they ever learnt was that of two-state systems in a finite-dimensional Hilbert space, and without ever acquiring even an inkling of ideas like radiation-matter interactions, transition rates, or the average decoherence times.)

The only way out, the only way that works in properly studying QM is this: Begin studying QM at a simpler level, finish developing as much understanding about its entire scope as possible (as happens in the typical Modern Physics courses), and then come to studying the same set of topics once again in a next iteration, but now to a greater depth. And, you have to keep repeating this process some 4–5 times. Often times, you have to come back from iteration n+2 to n.

As someone remarked at some forum (at Physics StackExchange or Quora or so), to learn QM, you have to give it “multiple passes.” Only then can you succeed understanding it. The idea of multiple passes has several implications. Let me mention only two of them. Both are specific to QM (and not to thermodynamics).

First, you have to develop the art of being able to hold some not-fully-satisfactory islands of understanding, with all the accompanying ambiguities, for extended periods of time (which usually runs into years!). You have to learn how to give a second or a third pass even when some of the things right from the first pass are still nowhere near getting clarified. You have to learn a lot of maths on the fly too. However, if you ask me, that’s a relatively easier task. The really difficult part is that you have to know (or learn!) how to keep forging ahead, even if at the same time, you carry a big set of nagging doubts that no one seems to know (or even care) about. (To make the matters worse, professional physicists, mathematicians and philosophers proudly keep telling you that these doubts will remain just as they are for the rest of your life.) You have to learn how to shove these ambiguous and un-clarified matters to some place near the back of your mind, you have to learn how to ignore them for a while, and still find the mental energy to once again begin right from the beginning, for your next pass: Planck and his cavity radiation, Einstein, blah blah blah blah blah!

Second, for the same reason (i.e. the necessity of multiple passes and the nature of QM), you also have to learn how to unlearn certain half-baked ideas and replace them later on with better ones. For a good example, go through Dan Styer’s paper on misconceptions about QM (listed near the end of this post).

Thus, two seemingly contradictory skills come into the play: You have to learn how to hold ambiguities without letting them affect your studies. At the same time, you also have to learn how not to hold on to them forever, or how to unlearn them, when the time to do becomes ripe.

Thus, learning QM does not involve just learning of new contents. You also have learn this art of building a sufficiently “temporary” but very complex conceptual structure in your mind—a structure that, despite all its complexity, still is resilient. You have to learn the art of holding such a framework together over a period of years, even as some parts of it are still getting replaced in your subsequent passes.

And, you have to compensate for all the failings of your teachers too (who themselves were told, effectively, to “shut up and calculate!”) Properly learning QM is a demanding enterprise.

# II. The list:

Now, with that long a preface, let me come to listing all the main books that I found especially helpful during my various passes. Please remember, I am still learning QM. I still don’t understand the second half of most any UG book on QM. This is a factual statement. I am not ashamed of it. It’s just that the first half itself managed to keep me so busy for so long that I could not come to studying, in an in-depth manner, the second half. (By the second half, I mean things like: the QM of molecules and binding, of their spectra, QM of solids, QM of complicated light-matter interactions, computational techniques like DFT, etc.) … OK. So, without any further ado, let me jot down the actual list.  I will subdivide it in several sub-sections

II.0. Junior-college (American high-school) level:

Obvious:

• Resnick and Halliday.
• Thomas and Finney. Also, Allan Jeffrey

II.1. Initial, college physics level:

• “Modern physics” by Beiser, or equivalent
• Optional but truly helpful: “Physical chemistry” by Atkins, or equivalent, i.e., only the parts relevant to QM. (I know engineers often tend to ignore the chemistry books, but they should not. In my experience, often times, chemistry books do a superior job of explaining physics. Physics, to paraphrase a witticism, is far too important to be left to the physicists!)

II.2. Preparatory material for some select topics:

• “Physics of waves” by Howard Georgi. Excellence written all over, but precisely for the same reason, take care to avoid the temptation to get stuck in it!
• Maths: No particular book, but a representative one would be Kreyszig, i.e., with Thomas and Finney or Allan Jeffrey still within easy reach.
• There are a few things you have to relearn, if necessary. These include: the idea of the limits of sequences and series. (Yes, go through this simple a topic too, once again. I mean it!). Then, the limits of functions.
Also try to relearn curve-tracing.
• Unlearn (or throw away) all the accounts of complex numbers which remain stuck at the level of how $\sqrt{-1}$ was stupefying, and how, when you have complex numbers, any arbitrary equation magically comes to have roots, etc. Unlearn all that talk. Instead, focus on the similarities of complex numbers to both the real numbers and vectors, and also their differences from each. Unlike what mathematicians love to tell you, complex numbers are not just another kind of numbers. They don’t represent just the next step in the logic of how the idea of numbers gets generalized as go from integers to real numbers. The reason is this: Unlike the integers, rationals, irrationals and reals, complex numbers take birth as composite numbers (as a pair of numbers that is ordered too), and they remain that way until the end of their life. Get that part right, and ignore all the mathematicians’ loose talk about it.
Study complex numbers in a way that, eventually, you should find yourself being comfortable with the two equivalent ways of modeling physical phenomena: as a set of two coupled real-valued differential equations, and as a single but complex-valued differential equation.
• Also try to become proficient with the two main expansions: the Taylor, and the Fourier.
• Also develop a habit of quickly substituting truncated expansions (i.e., either a polynomial, or a sum complex exponentials having just a few initial harmonics, not an entire infinity of them) into any “arbitrary” function as an ansatz, and see how the proposed theory pans out with these. The goal is to become comfortable, at the same time, with a habit of tracing conceptual pathways to the meaning of maths as well as with the computational techniques of FDM, FEM, and FFT.
• The finite differences approximation: Also, learn the art of quickly substituting the finite differences ($\Delta$‘s) in place of the differential quantities ($d$ or $\partial$) in a differential equation, and seeing how it pans out. The idea here is not just the computational modeling. The point is: Every differential equation has been derived in reference to an elemental volume which was then taken to a vanishingly small size. The variation of quantities of interest across such (infinitesimally small) volume are always represented using the Taylor series expansion.
(That’s correct! It is true that the derivations using the variational approach don’t refer to the Taylor expansion. But they also don’t use infinitesimal volumes; they refer to finite or infinite domains. It is the variation in functions which is taken to the vanishingly small limit in their case. In any case, if your derivation has an infinitesimall small element, bingo, you are going to use the Taylor series.)
Now, coming back to why you must learn develop the habit of having a finite differences approximation in place of a differential equation. The thing is this: By doing so, you are unpacking the derivation; you are traversing the analysis in the reverse direction, you are by the logic of the procedure forced to look for the physical (or at least lower-level, less abstract) referents of a mathematical relation/idea/concept.
While thus going back and forth between the finite differences and the differentials, also learn the art of tracing how the limiting process proceeds in each such a case. This part is not at all as obvious as you might think. It took me years and years to figure out that there can be infinitesimals within infinitesimals. (In fact, I have blogged about it several years ago here. More recently, I wrote a PDF document about how many numbers are there in the real number system, which discusses the same idea, from a different angle. In any case, if you were not shocked by the fact that there can be an infinity of infinitesimals within any infinitesimal, either think sufficiently long about it—or quit studying foundations of QM.)

II.3. Quantum chemistry level (mostly concerned with only the TISE, not TDSE):

• Optional: “QM: a conceptual approach” by Hameka. A fairly well-written book. You can pick it up for some serious reading, but also try to finish it as fast as you can, because you are going to relean the same stuff once again through the next book in the sequence. But yes, you can pick it up; it’s only about 200 pages.
• “Quantum chemistry” by McQuarrie. Never commit the sin of bypassing this excellent book.
A suggestion: Once you finish reading through this particular book, take a small (40 page) notebook, and write down (in the long hand) just the titles of the sections of each chapter of this book, followed by a listing of the important concepts / equations / proofs introduced in it. … You see, the section titles of this book themselves are complete sentences that encapsulate very neat nuggets. Here are a couple of examples: “5.6: The harmonic oscillator accounts for the infrared spectrum of a diatomic molecule.” Yes, that’s a section title! Here is another: “6.2: If a Hamiltonian is separable, then its eigenfunctions are products of simpler eigenfunctions.” See why I recommend this book? And this (40 page notebook) way of studying it?
• “Quantum physics of atoms, molecules, solids, nuclei, and particles” (yes, that’s the title of this single volume!) by Eisberg and Resnick. This Resnick is the same one as that of Resnick and Halliday. Going through the same topics via yet another thick book (almost 850 pages) can get exasperating, at least at times. But guess if you show some patience here, it should simplify things later. …. Confession: I was too busy with teaching and learning engineering topics like FEM, CFD, and also with many other things in between. So, I could not find the time to read this book the way I would have liked to. But from whatever I did read (and I did go over a fairly good portion of it), I can tell you that not finishing this book was a mistake on my part. Don’t repeat my mistake. Further, I do keep going back to it, and may be as a result, I would one day have finished it! One more point. This book is more than quantum chemistry; it does discuss the time-dependent parts too. The only reason I include it in this sub-section (chemistry) rather than the next (physics) is because the emphasis here is much more on TISE than TDSE.

II.4. Quantum physics level (includes TDSE):

• “Quantum physics” by Alastair I. M. Rae. Hands down, the best book in its class. To my mind, it easily beats all of the following: Griffiths, Gasiorowicz, Feynman, Susskind, … .
Oh, BTW, this is the only book I have ever come across which does not put scare-quotes around the word “derivation,” while describing the original development of the Schrodinger equation. In fact, this text goes one step ahead and explicitly notes the right idea, viz., that Schrodinger’s development is a derivation, but it is an inductive derivation, not deductive. (… Oh God, these modern American professors of physics!)
But even leaving this one (arguably “small”) detail aside, the book has excellence written all over it. Far better than the competition.
Another attraction: The author touches upon all the standard topics within just about 225 pages. (He also has further 3 chapters, one each on relativity and QM, quantum information, and conceptual problems with QM. However, I have mostly ignored these.) When a book is of manageable size, it by itself is an overload reducer. (This post is not a portion from a text-book!)
The only “drawback” of this book is that, like many British authors, Rae has a tendency to seamlessly bunch together a lot of different points into a single, bigger, paragraph. He does not isolate the points sufficiently well. So, you have to write a lot of margin notes identifying those distinct, sub-paragraph level, points. (But one advantage here is that this procedure is very effective in keeping you glued to the book!)
• “Quantum physics” by Griffiths. Oh yes, Griffiths is on my list too. It’s just that I find it far better to go through Rae first, and only then come to going through Griffiths.
• … Also, avoid the temptation to read both these books side-by-side. You will soon find that you can’t do that. And so, driven by what other people say, you will soon end up ditching Rae—which would be a grave mistake. Since you can keep going through only one of them, you have to jettison the other. Here, I would advise you to first complete Rae. It’s indispensable. Griffiths is good too. But it is not indispensable. And as always, if you find the time and the inclination, you can always come back to Griffiths.

Starting sometime after finishing the initial UG quantum chemistry level books, but preferably after the quantum physics books, use the following two:

• “Foundations of quantum mechanics” by Travis Norsen. Very, very good. See my “review” here [^]
• “Foundations of quantum mechanics: from photons to quantum computers” by Reinhold Blumel.
Just because people don’t rave a lot about this book doesn’t mean that it is average. This book is peculiar. It does look very average if you flip through all its pages within, say, 2–3 minutes. But it turns out to be an extraordinarily well written book once you begin to actually read through its contents. The coverage here is concise, accurate, fairly comprehensive, and, as a distinctive feature, it also is fairly up-to-date.
Unlike the other text-books, Blumel gives you a good background in the specifics of the modern topics as well. So, once you complete this book, you should find it easy (to very easy) to understand today’s pop-sci articles, say those on quantum computers. To my knowledge, this is the only text-book which does this job (of introducing you to the topics that are relevant to today’s research), and it does this job exceedingly well.
• Use Blumel to understand the specifics, and use Norsen to understand their conceptual and the philosophical underpinnings.

II.Appendix: Miscellaneous—no levels specified; figure out as you go along:

• “Schrodinger’s cat” by John Gribbin. Unquestionably, the best pop-sci book on QM. Lights your fire.
• “Quantum” by Manjit Kumar. Helps keep the fire going.
• Kreyszig or equivalent. You need to master the basic ideas of the Fourier theory, and of solutions of PDEs via the separation ansatz.
• However, for many other topics like spherical harmonics or calculus of variations, you have to go hunting for explanations in some additional books. I “learnt” the spherical harmonics mostly through some online notes (esp. those by Michael Fowler of Univ. of Virginia) and QM textbooks, but I guess that a neat exposition of the topic, couched in contexts other than QM, would have been helpful. May be there is some ancient acoustics book that is really helpful. Anyway, I didn’t pursue this topic to any great depth (in fact I more or less skipped over it) because as it so happens, analytical methods fall short for anything more complex than the hydrogenic atoms.
• As to the variational calculus, avoid all the physics and maths books like a plague! Instead, learn the topic through the FEM books. Introductory FEM books have become vastly (i.e. categorically) better over the course of my generation. Today’s FEM text-books do provide a clear evidence that the authors themselves know what they are talking about! Among these books, just for learning the variational calculus aspects, I would advise going through Seshu or Fish and Belytschko first, and then through the relevant chapter from Reddy‘s book on FEM. In any case, avoid Bathe, Zienkiewicz, etc.; they are too heavily engineering-oriented, and often, in general, un-necessarily heavy-duty (though not as heavy-duty as Lancosz). Not very suitable for learning the basics of CoV as is required in the UG QM. A good supplementary book covering CoV is noted next.
• “From calculus to chaos: an introduction to dynamics” by David Acheson. A gem of a book. Small (just about 260 pages, including program listings—and just about 190 pages if you ignore them.) Excellent, even if, somehow, it does not appear on people’s lists. But if you ask me, this book is a must read for any one who has anything to do with physics or engineering. Useful chapters exist also on variational calculus and chaos. Comes with easy to understand QBasic programs (and their updated versions, ready to run on today’s computers, are available via the author’s Web site). Wish it also had chapters, say one each, on the mechanics of materials, and on fracture mechanics.
• Linear algebra. Here, keep your focus on understanding just the two concepts: (i) vector spaces, and (ii) eigen-vectors and -values. Don’t worry about other topics (like LU decomposition or the power method). If you understand these two topics right, the rest will follow “automatically,” more or less. To learn these two topics, however, don’t refer to text-books (not even those by Gilbert Strang or so). Instead, google on the online tutorials on computer games programming. This way, you will come to develop a far better (even robust) understanding of these concepts. … Yes, that’s right. One or two games programmers, I very definitely remember, actually did a much superior job of explaining these ideas (with all their complexity) than what any textbook by any university professor does. (iii) Oh yes, BTW, there is yet another concept which you should learn: “tensor product”. For this topic, I recommend Prof. Zhigang Suo‘s notes on linear algebra, available off iMechanica. These notes are a work in progress, but they are already excellent even in their present form.
• Probability. Contrary to a wide-spread impression (and to what one group of QM interpreters say), you actually don’t need much of statistics or probability in order to get the essence of QM right. Whatever you need has already been taught to you in your UG engineering/physics courses.Personally, though I haven’t yet gone through them, the two books on my radar (more from the data science angle) are: “Elementary probability” by Stirzaker, and “All of statistics” by Wasserman. But, frankly speaking, as far as QM itself is concerned, your intuitive understanding of probability as developed through your routine UG courses should be enough, IMHO.
• As to AJP type of articles, go through Dan Styer‘s paper on the nine formulations (doi:10.1119/1.1445404). But treat his paper on the common misconceptions (10.1119/1.18288) with a bit of caution; some of the ideas he lists as “misconceptions” are not necessarily so.
• arXiv tutorials/articles: Sometime after finishing quantum chemistry and before beginning quantum physics, go through the tutorial on QM by Bram Gaasbeek [^]. Neat, small, and really helpful for self-studies of QM. (It was written when the author was still a student himself.) Also, see the article on the postulates by Dorabantu [^]. Definitely helpful. Finally, let me pick up just one more arXiv article: “Entanglement isn’t just for spin” by Dan Schroeder [^]. Comes with neat visualizations, and helps demystify entanglement.
• Computational physics: Several good resources are available. One easy to recommend text-book is the one by Landau, Perez and Bordeianu. Among the online resources, the best collection I found was the one by Ian Cooper (of Univ. of Sydney) [^]. He has only MatLab scripts, not Python, but they all are very well documented (in an exemplary manner) via accompanying PDF files. It should be easy to port these programs to the Python eco-system.

Yes, we (finally) are near the end of this post, so let me add the mandatory catch-all clauses: This list is by no means comprehensive! This list supersedes any other list I may have put out in the past. This list may undergo changes in future.

Done.

OK. A couple of last minute addenda: For contrast, see the article “What is the best textbook for self-studying quantum mechanics?” which has appeared, of all places, on the Forbes!  [^]. (Looks like the QC-related hype has found its way into the business circles as well!) Also see the list at BookScrolling.com: “The best books to learn about quantum physics” [^].

OK. Now, I am really done.

A song I like:
(Marathi) “kiteedaa navyaane tulaa aaThavaave”
Music: Mandar Apte
Singer: Mandar Apte. Also, a separate female version by Arya Ambekar
Lyrics: Devayani Karve-Kothari

[Arya Ambekar’s version is great too, but somehow, I like Mandar Apte’s version better. Of course, I do often listen to both the versions. Excellent.]

[Almost 5000 More than 5,500 words! Give me a longer break for this time around, a much longer one, in fact… In the meanwhile, take care and bye until then…]