Machine “Learning”—An Entertainment [Industry] Edition

Yes, “Machine ‘Learning’,” too, has been one of my “research” interests for some time by now. … Machine learning, esp. ANN (Artificial Neural Networks), esp. Deep Learning. …

Yesterday, I wrote a comment about it at iMechanica. Though it was made in a certain technical context, today I thought that the comment could, perhaps, make sense to many of my general readers, too, if I supply a bit of context to it. So, let me report it here (after a bit of editing). But before coming to my comment, let me first give you the context in which it was made:


Context for my iMechanica comment:

It all began with a fellow iMechanician, one Mingchuan Wang, writing a post of the title “Is machine learning a research priority now in mechanics?” at iMechanica [^]. Biswajit Banerjee responded by pointing out that

“Machine learning includes a large set of techniques that can be summarized as curve fitting in high dimensional spaces. [snip] The usefulness of the new techniques [in machine learning] should not be underestimated.” [Emphasis mine.]

Then Biswajit had pointed out an arXiv paper [^] in which machine learning was reported as having produced some good DFT-like simulations for quantum mechanical simulations, too.

A word about DFT for those who (still) don’t know about it:

DFT, i.e. Density Functional Theory, is “formally exact description of a many-body quantum system through the density alone. In practice, approximations are necessary” [^]. DFT thus is a computational technique; it is used for simulating the electronic structure in quantum mechanical systems involving several hundreds of electrons (i.e. hundreds of atoms). Here is the obligatory link to the Wiki [^], though a better introduction perhaps appears here [(.PDF) ^]. Here is a StackExchange on its limitations [^].

Trivia: Kohn and Sham received a Physics Nobel for inventing DFT. It was a very, very rare instance of a Physics Nobel being awarded for an invention—not a discovery. But the Nobel committee, once again, turned out to have put old Nobel’s money in the right place. Even if the work itself was only an invention, it did directly led to a lot of discoveries in condensed matter physics! That was because DFT was fast—it was fast enough that it could bring the physics of the larger quantum systems within the scope of (any) study at all!

And now, it seems, Machine Learning has advanced enough to be able to produce results that are similar to DFT, but without using any QM theory at all! The computer does have to “learn” its “art” (i.e. “skill”), but it does so from the results of previous DFT-based simulations, not from the theory at the base of DFT. But once the computer does that—“learning”—and the paper shows that it is possible for computer to do that—it is able to compute very similar-looking simulations much, much faster than even the rather fast technique of DFT itself.

OK. Context over. Now here in the next section is my yesterday’s comment at iMechanica. (Also note that the previous exchange on this thread at iMechanica had occurred almost a year ago.) Since it has been edited quite a bit, I will not format it using a quotation block.


[An edited version of my comment begins]

A very late comment, but still, just because something struck me only this late… May as well share it….

I think that, as Biswajit points out, it’s a question of matching a technique to an application area where it is likely to be of “good enough” a fit.

I mean to say, consider fluid dynamics, and contrast it to QM.

In (C)FD, the nonlinearity present in the advective term is a major headache. As far as I can gather, this nonlinearity has all but been “proved” as the basic cause behind the phenomenon of turbulence. If so, using machine learning in CFD would be, by the simple-minded “analysis”, a basically hopeless endeavour. The very idea of using a potential presupposes differential linearity. Therefore, machine learning may be thought as viable in computational Quantum Mechanics (viz. DFT), but not in the more mundane, classical mechanical, CFD.

But then, consider the role of the BCs and the ICs in any simulation. It is true that if you don’t handle nonlinearities right, then as the simulation time progresses, errors are soon enough going to multiply (sort of), and lead to a blowup—or at least a dramatic departure from a realistic simulation.

But then, also notice that there still is some small but nonzero interval of time which has to pass before a really bad amplification of the errors actually begins to occur. Now what if a new “BC-IC” gets imposed right within that time-interval—the one which does show “good enough” an accuracy? In this case, you can expect the simulation to remain “sufficiently” realistic-looking for a long, very long time!

Something like that seems to have been the line of thought implicit in the results reported by this paper: [(.PDF) ^].

Machine learning seems to work even in CFD, because in an interactive session, a new “modified BC-IC” is every now and then is manually being introduced by none other than the end-user himself! And, the location of the modification is precisely the region from where the flow in the rest of the domain would get most dominantly affected during the subsequent, small, time evolution.

It’s somewhat like an electron rushing through a cloud chamber. By the uncertainty principle, the electron “path” sure begins to get hazy immediately after it is “measured” (i.e. absorbed and re-emitted) by a vapor molecule at a definite point in space. The uncertainty in the position grows quite rapidly. However, what actually happens in a cloud chamber is that, before this cone of haziness becomes too big, comes along another vapor molecule, and “zaps” i.e. “measures” the electron back on to a classical position. … After a rapid succession of such going-hazy-getting-zapped process, the end result turns out to be a very, very classical-looking (line-like) path—as if the electron always were only a particle, never a wave.

Conclusion? Be realistic about how smart the “dumb” “curve-fitting” involved in machine learning can at all get. Yet, at the same time, also remain open to all the application areas where it can be made it work—even including those areas where, “intuitively”, you wouldn’t expect it to have any chance to work!

[An edited version of my comment is over. Original here at iMechanica [^]]


 

“Boy, we seem to have covered a lot of STEM territory here… Mechanics, DFT, QM, CFD, nonlinearity. … But where is either the entertainment or the industry you had promised us in the title?”

You might be saying that….

Well, the CFD paper I cited above was about the entertainment industry. It was, in particular, about the computer games industry. Go check out SoHyeon Jeong’s Web site for more cool videos and graphics [^], all using machine learning.


And, here is another instance connected with entertainment, even though now I am going to make it (mostly) explanation-free.

Check out the following piece of art—a watercolor landscape of a monsoon-time but placid sea-side, in fact. Let me just say that a certain famous artist produced it; in any case, the style is plain unmistakable. … Can you name the artist simply by looking at it? See the picture below:

A sea beach in the monsoons. Watercolor.

If you are unable to name the artist, then check out this story here [^], and a previous story here [^].


A Song I Like:

And finally, to those who have always loved Beatles’ songs…

Here is one song which, I am sure, most of you had never heard before. In any case, it came to be distributed only recently. When and where was it recorded? For both the song and its recording details, check out this site: [^]. Here is another story about it: [^]. And, if you liked what you read (and heard), here is some more stuff of the same kind [^].


Endgame:

I am of the Opinion that 99% of the “modern” “artists” and “music composers” ought to be replaced by computers/robots/machines. Whaddya think?

[Credits: “Endgame” used to be the way Mukul Sharma would end his weekly Mindsport column in the yesteryears’ Sunday Times of India. (The column perhaps also used to appear in The Illustrated Weekly of India before ToI began running it; at least I have a vague recollection of something of that sort, though can’t be quite sure. … I would be a school-boy back then, when the Weekly perhaps ran it.)]

 

Advertisements

Micro-level water-resources engineering—8: Measure that water evaporation! Right now!!

It’s past the middle of May—the hottest time of the year in India.

The day-time is still lengthening. And it will continue doing so well up to the summer solstice in the late June, though once monsoon arrives some time in the first half of June, the solar flux in this part of the world would get reduced due to the cloud cover, and so, any further lengthening of the day would not matter.

In the place where I these days live, the day-time temperature easily goes up to 42–44 deg. C. This high a temperature is, that way, not at all unusual for most parts of Maharashtra; sometimes Pune, which is supposed to be a city of a pretty temperate climate (mainly because of the nearby Sahyaadris), also registers the max. temperatures in the early 40s. But what makes the region where I currently live worse than Pune are these two factors: (i) the minimum temperature too stays as high as 30–32 deg. C here whereas in Pune it could easily be falling to 27–26 deg. C even during May, and (ii) the fall of the temperatures at night-time proceeds very gradually here. On a hot day, it can easily be as high as 38 deg C. even after the sunset, and even 36–37 deg. C right by the time it’s the mid-night; the drop below 35 deg. C occurs only for the 3–4 hours in the early morning, between 4 to 7 AM. In comparison, Pune is way cooler. The max. temperatures Pune registers may be similar, but the evening- and the night-time temperatures fall down much more rapidly there.

There is a lesson for the media here. Media obsesses over the max. temperature (and its record, etc.). That’s because the journos mostly are BAs. (LOL!) But anyone who has studied physics and calculus knows that it’s the integral of temperature with respect to time that really matters, because it is this quantity which scales with the total thermal energy transferred to a body. So, the usual experience common people report is correct. Despite similar max. temperatures, this place is hotter, much hotter than Pune.


And, speaking of my own personal constitution, I can handle a cold weather way better than I can handle—if at all I can handle—a hot weather. [Yes, in short, I’ve been in a bad shape for the past month or more. Lethargic. Lackadaisical. Enervated. You get the idea.]


But why is it that the temperature does not matter as much as the thermal energy does?

Consider a body, say a cube of metal. Think of some hypothetical apparatus that keeps this body at the same cool temperature at all times, say, at 20 deg. C.  Here, choose the target temperature to be lower than the minimum temperature in the day. Assume that the atmospheric temperature at two different places varies between the same limits, say, 42 to 30 deg. C. Since the target temperature is lower than the minimum ambient temperature, you would have to take heat out of the cube at all times.

The question is, at which of the two places the apparatus has to work harder. To answer that question, you have to calculate the total thermal energy that has be drained out of the cube over a single day. To answer this second question, you would need the data of not just the lower and upper limits of the temperature but also how it varies with time between two limits.


The humidity too is lower here as compared to in Pune (and, of course, in Mumbai). So, it feels comparatively much more drier. It only adds to the real feel of a real hot weather.

One does not realize it, but the existence of a prolonged high temperature makes the atmosphere here imperceptibly slowly but also absolutely insurmountably, dehydrating.

Unlike in Mumbai, one does not notice much perspiration here, and that’s because the air is so dry that any perspiration that does occur also dries up very fast. Shirts getting drenched by perspiration is not a very common sight here. Overall, desiccating would be the right word to describe this kind of an air.

So, yes, it’s bad, but you can always take precautions. Make sure to drink a couple of glasses of cool water (better still, fresh lemonade) before you step out—whether you are thirsty or not. And take an onion with you when you go out; if you begin to feel too much of heat, you can always crush the onion with hand and apply the juice onto the top of your head. [Addendum: A colleague just informed me that it’s even better to actually cut the onion and keep its cut portion touching to your body, say inside your shirt. He has spent summers in eastern Maharashtra, where temperatures can reach 47 deg. C. … Oh well!]

Also, eat a lot more onions than you normally do.

And, once you return home, make sure not to drink water immediately. Wait for 5–10 minutes. Otherwise, the body goes into a shock, and the ensuing transient spikes in your biological metabolism can, at times, even trigger the sun-stroke—which can even be fatal. A simple precaution helps avoid it.

For the same reason, take care to sit down in the shade of a tree for a few minutes before you eat that slice of water-melon. Water-melon is nothing but more than 95% water, thrown with a little sugar, some fiber, and a good measure of minerals. All in all, good for your body because even if the perspiration is imperceptible in the hot and dry regions, it is still occurring, and with it, the body is being drained of the necessary electrolytes and minerals. … Lemonades and water-melons supply the electrolytes and the minerals. People do take care not to drink lemonade in the Sun, but they don’t always take the same precaution for water-melon. Yet, precisely because a water-melon has so much water, you should take care not to expose your body to a shock. [And, oh, BTW, just in case you didn’t know already, the doctor-recommended alternative to Electral powder is: your humble lemonade! Works exactly equivalently!!]


Also, the very low levels of humidity also imply that in places like this, the desert-cooler is effective, very effective. The city shops are full of them. Some of these air-coolers sport a very bare-bones design. Nothing fancy like the Symphony Diet cooler (which I did buy last year in Pune!). The air-coolers locally made here can be as simple as just an open tray at the bottom to hold the water, a cube made of a coarse wire-mesh which is padded with the khus/wood sheathings curtain, and a robust fan operating [[very] noisily]. But it works wonderfully. And these local-made air-coolers also are very inexpensive. You can get one for just Rs. 2,500 or 3,000. I mean the ones which have a capacity to keep at least 3–4 people cool.(Branded coolers like the one I bought in Pune—and it does work even in Pune—often go above Rs. 10,000. [I bought that cooler last year because I didn’t have a job, thanks to the Mechanical Engineering Professors in the Savitribai Phule Pune University.])


That way, I also try to think of the better things this kind of an air brings. How the table salt stays so smoothly flowing, how the instant coffee powder or Bournvita never turns into a glue, how an opened packet of potato chips stays so crisp for days, how washed clothes dry up in no time…

Which, incidentally, brings me to the topic of this post.


The middle—or the second half—of May also is the most ideal time to conduct evaporation experiments.

If you are looking for a summer project, here is one: to determine the evaporation rate in your locality.

Take a couple of transparent plastic jars of uniform cross section. The evaporation rate is not very highly sensitive to the cross-sectional area, but it does help to take a vessel or a jar of sizeable diameter.

Affix a mm scale on the outside of each jar, say using cello-tape. Fill the plastic jars to some level almost to the full.

Keep one jar out in the open (exposed to the Sun), and another one, inside your home, in the shade. For the jar kept outside, make sure that birds don’t come and drink the water, thereby messing up with your measurements. For this purpose, you may surround the jar with an enclosure having a coarse mesh. The mesh must be coarse; else it will reduce the solar flux. The “reduction in the solar flux” is just a fancy [mechanical [thermal] engineering] term for saying that the mesh, if too fine, might cast too significant a shadow.

Take measurements of the heights of the water daily at a fixed time of the day, say at 6:00 PM. Conduct the experiment for a week or 10 days.

Then, plot a graph of the daily water level vs. the time elapsed, for each jar.

Realize, the rate of evaporation is measured in terms of the fall in the height, and not in terms of the volume of water lost. That’s because once the exposed area is bigger than some limit, the evaporation rate (the loss in height) is more or less independent of the cross-sectional area.

Now figure out:

Does the evaporation rate stay the same every day? If there is any significant departure from a straight-line graph, how do you explain it? Was there a measurement error? Was there an unusually strong wind on a certain day? a cloud cover?

Repeat the experiment next winter (around the new year), and determine the rate of evaporation at that time.

Later on, also make some calculations. If you are building a check-dam or a farm-pond, how much would be the evaporation loss over the five months from January to May-end? Is the height of your water storage system enough to make it practically useful? economically viable?


A Song I Like:

(Hindi) “mausam aayegaa, jaayegaa, pyaar sadaa muskuraayegaa…”
Music: Manas Mukherjee
Singers: Manna Dey and Asha Bhosale
Lyrics: Vithalbhai Patel

See, how hard I am trying to become an Approved (Full) Professor of Mechanical Engineering in SPPU?—4

In this post, I provide my answer to the question which I had raised last time, viz., about the differences between the \Delta, the \text{d}, and the \delta (the first two, of the usual calculus, and the last one, of the calculus of variations).


Some pre-requisite ideas:

A system is some physical object chosen (or isolated) for study. For continua, it is convenient to select a region of space for study, in which case that region of space (holding some physical continuum) may also be regarded as a system. The system boundary is an abstraction.

A state of a system denotes a physically unique and reproducible condition of that system. State properties are the properties or attributes that together uniquely and fully characterize a state of a system, for the chosen purposes. The state is an axiom, and state properties are its corollary.

State properties for continua are typically expressed as functions of space and time. For instance, pressure, temperature, volume, energy, etc. of a fluid are all state properties. Since state properties uniquely define the condition of a system, they represent definite points in an appropriate, abstract, (possibly) higher-dimensional state space. For this reason, state properties are also called point functions.

A process (synonymous to system evolution) is a succession of states. In classical physics, the succession (or progression) is taken to be continuous. In quantum mechanics, there is no notion of a process; see later in this post.

A process is often represented as a path in a state space that connects the two end-points of the staring and ending states. A parametric function defined over the length of a path is called a path function.

A cyclic process is one that has the same start and end points.

During a cyclic process, a state function returns to its initial value. However, a path function does not necessarily return to the same value over every cyclic change—it depends on which particular path is chosen. For instance, if you take a round trip from point A to point B and back, you may spend some amount of money m if you take one route but another amount n if you take another route. In both cases you do return to the same point viz. A, but the amount you spend is different for each route. Your position is a state function, and the amount you spend is a path function.

[I may make the above description a bit more rigorous later on (by consulting a certain book which I don’t have handy right away (and my notes of last year are gone in the HDD crash)).]


The \Delta, the \text{d}, and the \delta:

The \Delta denotes a sufficiently small but finite, and locally existing difference in different parts of a system. Typically, since state properties are defined as (continuous) functions of space and time, what the \Delta represents is a finite change in some state property function that exists across two different but adjacent points in space (or two nearby instants in times), for a given system.

The \Delta is a local quantity, because it is defined and evaluated around a specific point of space and/or time. In other words, an instance of \Delta is evaluated at a fixed x or t. The \Delta x simply denotes a change of position; it may or may not mean a displacement.

The \text{d} (i.e. the infinitesimal) is nothing but the \Delta taken in some appropriate limiting process to the vanishingly small limit.

Since \Delta is locally defined, so is the infinitesimal (i.e. \text{d}).

The \delta of CoV is completely different from the above two concepts.

The \delta is a sufficiently small but global difference between the states (or paths) of two different, abstract, but otherwise identical views of the same physically existing system.

Considering the fact that an abstract view of a system is itself a system, \delta also may be regarded as a difference between two systems.

Though differences in paths are not only possible but also routinely used in CoV, in this post, to keep matters simple, we will mostly consider differences in the states of the two systems.

In CoV, the two states (of the two systems) are so chosen as to satisfy the same Dirichlet (i.e. field) boundary conditions separately in each system.

The state function may be defined over an abstract space. In this post, we shall not pursue this line of thought. Thus, the state function will always be a function of the physical, ambient space (defined in reference to the extensions and locations of concretely existing physical objects).

Since a state of a system of nonzero size can only be defined by specifying its values for all parts of a system (of which it is a state), a difference between states (of the two systems involved in the variation \delta) is necessarily global.

In defining \delta, both the systems are considered only abstractly; it is presumed that at most one of them may correspond to an actual state of a physical system (i.e. a system existing in the physical reality).

The idea of a process, i.e. the very idea of a system evolution, necessarily applies only to a single system.

What the \delta represents is not an evolution because it does not represent a change in a system, in the first place. The variation, to repeat, represents a difference between two systems satisfying the same field boundary conditions. Hence, there is no evolution to speak of. When compressed air is passed into a rubber balloon, its size increases. This change occurs over certain time, and is an instance of an evolution. However, two rubber balloons already inflated to different sizes share no evolutionary relation with each other; there is no common physical process connecting the two; hence no change occurring over time can possibly enter their comparative description.

Thus, the “change” denoted by \delta is incapable of representing a process or a system evolution. In fact, the word “change” itself is something of a misnomer here.

Text-books often stupidly try to capture the aforementioned idea by saying that \delta represents a small and possibly finite change that occurs without any elapse of time. Apart from the mind-numbing idea of a finite change occurring over no time (or equally stupefying ideas which it suggests, viz., a change existing at literally the same instant of time, or, alternatively, a process of change that somehow occurs to a given system but “outside” of any time), what they, in a way, continue to suggest also is the erroneous idea that we are working with only a single, concretely physical system, here.

But that is not the idea behind \delta at all.

To complicate the matters further, no separate symbol is used when the variation \delta is made vanishingly small.

In the primary sense of the term variation (or \delta), the difference it represents is finite in nature. The variation is basically a function of space (and time), and at every value of x (and t), the value of \delta is finite, in the primary sense of the word. Yes, these values can be made vanishingly small, though the idea of the limits applied in this context is different. (Hint: Expand each of the two state functions in a power series and relate each of the corresponding power terms via a separate parameter. Then, put the difference in each parameter through a limiting process to vanish. You may also use the Fourier expansion.))

The difference represented by \delta is between two abstract views of a system. The two systems are related only in an abstract view, i.e., only in (the mathematical) thought. In the CoV, they are supposed as connected, but the connection between them is not concretely physical because there are no two separate physical systems concretely existing, in the first place. Both the systems here are mathematical abstractions—they first have been abstracted away from the real, physical system actually existing out there (of which there is only a single instance).

But, yes, there is a sense in which we can say that \delta does have a physical meaning: it carries the same physical units as for the state functions of the two abstract systems.


An example from biology:

Here is an example of the differences between two different paths (rather than two different states).

Plot the height h(t) of a growing sapling at different times, and connect the dots to yield a continuous graph of the height as a function of time. The difference in the heights of the sapling at two different instants is \Delta h. But if you consider two different saplings planted at the same time, and assuming that they grow to the same final height at the end of some definite time period (just pick some moment where their graphs cross each other), and then, abstractly regarding them as some sort of imaginary plants, if you plot the difference between the two graphs, that is the variation or \delta h(t) in the height-function of either. The variation itself is a function (here of time); it has the units, of course, of m.


Summary:

The \Delta is a local change inside a single system, and \text{d} is its limiting value, whereas the \delta is a difference across two abstract systems differing in their global states (or global paths), and there is no separate symbol to capture this object in the vanishingly small limit.


Exercises:

Consider one period of the function y = A \sin(x), say over the interval [0,2\pi]; A = a is a small, real-valued, constant. Now, set A = 1.1a. Is the change/difference here a \delta or a \Delta? Why or why not?

Now, take the derivative, i.e., y' = A \cos(x), with A = a once again. Is the change/difference here a \delta or a \Delta? Why or why not?

Which one of the above two is a bigger change/difference?

Also consider this angle: Taking the derivative did affect the whole function. If so, why is it that we said that \text{d} was necessarily a local change?


An important and special note:

The above exercises, I am sure, many (though not all) of the Officially Approved Full Professors of Mechanical Engineering at the Savitribai Phule Pune University and COEP would be able to do correctly. But the question I posed last time was: Would it be therefore possible for them to spell out the physical meaning of the variation i.e. \delta? I continue to think not. And, importantly, even among those who do solve the above exercises successfully, they wouldn’t be too sure about their own answers. Upon just a little deeper probing, they would just throw up their hands. [Ditto, for many American physicists.] Even if a conceptual clarity is required in applications.

(I am ever willing and ready to change my mind about it, but doing so would need some actual evidence—just the way my (continuing) position had been derived, in the first place, from actual observations of them.)

The reason I made this special note was because I continue to go jobless, and nearly bank balance-less (and also, nearly cashless). And it all is basically because of folks like these (and the Indians like the SPPU authorities). It is their fault. (And, no, you can’t try to lift what is properly their moral responsibility off their shoulders and then, in fact, go even further, and attempt to place it on mine. Don’t attempt doing that.)


A Song I Like:

[May be I have run this song before. If yes, I will replace it with some other song tomorrow or so. No I had not.]

Hindi: “Thandi hawaa, yeh chaandani suhaani…”
Music and Singer: Kishore Kumar
Lyrics: Majrooh Sultanpuri

[A quick ‘net search on plagiarism tells me that the tune of this song was lifted from Julius La Rosa’s 1955 song “Domani.” I heard that song for the first time only today. I think that the lyrics of the Hindi song are better. As to renditions, I like Kishor Kumar’s version better.]


[Minor editing may be done later on and the typos may be corrected, but the essentials of my positions won’t be. Mostly done right today, i.e., on 06th January, 2017.]

[E&OE]

 

See, how hard I am trying to become an Approved (Full) Professor of Mechanical Engineering in SPPU?—3

I was looking for a certain book on heat transfer which I had (as usual) misplaced somewhere, and while searching for that book at home, I accidentally ran into another book I had—the one on Classical Mechanics by Rana and Joag [^].

After dusting this book a bit, I spent some time in one typical way, viz. by going over some fond memories associated with a suddenly re-found book…. The memories of how enthusiastic I once was when I had bought that book; how I had decided to finish that book right within weeks of buying it several years ago; the number of times I might have picked it up, and soon later on, kept it back aside somewhere, etc.  …

Yes, that’s right. I have not yet managed to finish this book. Why, I have not even managed to begin reading this book the way it should be read—with a paper and pencil at hand to work through the equations and the problems. That was the reason why, I now felt a bit guilty. … It just so happened that it was just the other day (or so) when I was happily mentioning the Poisson brackets on Prof. Scott Aaronson’s blog, at this thread [^]. … To remove (at least some part of) my sense of guilt, I then decided to browse at least through this part (viz., Poisson’s brackets) in this book. … Then, reading a little through this chapter, I decided to browse through the preceding chapters from the Lagrangian mechanics on which it depends, and then, in general, also on the calculus of variations.

It was at this point that I suddenly happened to remember the reason why I had never been able to finish (even the portions relevant to engineering from) this book.

The thing was, the explanation of the \delta—the delta of the variational calculus.

The explanation of what the \delta basically means, I had found right back then (many, many years ago), was not satisfactorily given in this book. The book did talk of all those things like the holonomic constraints vs. the nonholonomic constraints, the functionals, integration by parts, etc. etc. etc. But without ever really telling me, in a forth-right and explicit manner, what the hell this \delta was basically supposed to mean! How this \delta y was different from the finite changes (\Delta y) and the infinitesimal changes (\text{d}y) of the usual calculus, for instance. In terms of its physical meaning, that is. (Hell, this book was supposed to be on physics, wasn’t it?)

Here, I of course fully realize that describing Rana and Joag’s book as “unsatisfactory” is making a rather bold statement, a very courageous one, in fact. This book is extraordinarily well-written. And yet, there I was, many, many years ago, trying to understand the delta, and not getting anywhere, not even with this book in my hand. (OK, a confession. The current copy which I have is not all that old. My old copy is gone by now (i.e., permanently misplaced or so), and so, the current copy is the one which I had bought once again, in 2009. As to my old copy, I think, I had bought it sometime in the mid-1990s.)

It was many years later, guess some time while teaching FEM to the undergraduates in Mumbai, that the concept had finally become clear enough to me. Most especially, while I was going through P. Seshu’s and J. N. Reddy’s books. [Reflected Glory Alert! Professor P. Seshu was my class-mate for a few courses at IIT Madras!] However, even then, even at that time, I remember, I still had this odd feeling that the physical meaning was still not clear to me—not as as clear as it should be. The matter eventually became “fully” clear to me only later on, while musing about the differences between the perspective of Thermodynamics on the one hand and that of Heat Transfer on the other. That was some time last year, while teaching Thermodynamics to the PG students here in Pune.

Thermodynamics deals with systems at equilibria, primarily. Yes, its methods can be extended to handle also the non-equilibrium situations. However, even then, the basis of the approach summarily lies only in the equilibrium states. Heat Transfer, on the other hand, necessarily deals with the non-equilibrium situations. Remove the temperature gradient, and there is no more heat left to speak of. There does remain the thermal energy (as a form of the internal energy), but not heat. (Remember, heat is the thermal energy in transit that appears on a system boundary.) Heat transfer necessarily requires an absence of thermal equilibrium. … Anyway, it was while teaching thermodynamics last year, and only incidentally pondering about its differences from heat transfer, that the idea of the variations (of Cov) had finally become (conceptually) clear to me. (No, CoV does not necessarily deal only with the equilibrium states; it’s just that it was while thinking about the equilibrium vs. the transient that the matter about CoV had suddenly “clicked” to me.)

In this post, let me now note down something on the concept of the variation, i.e., towards understanding the physical meaning of the symbol \delta.

Please note, I have made an inline update on 26th December 2016. It makes the presentation of the calculus of variations a bit less dumbed down. The updated portion is clearly marked as such, in the text.


The Problem Description:

The concept of variations is abstract. We would be better off considering a simple, concrete, physical situation first, and only then try to understand the meaning of this abstract concept.

Accordingly, consider a certain idealized system. See its schematic diagram below:

mechanicalengineering_1d_cov

 

 

 

 

There is a long, rigid cylinder made from some transparent material like glass. The left hand-side end of the cylinder is hermetically sealed with a rigid seal. At the other end of the cylinder, there is a friction-less piston which can be driven by some external means.

Further, there also are a couple of thin, circular, piston-like disks (D_1 and D_2) placed inside the cylinder, at some x_1 and x_2 positions along its length. These disks thus divide the cylindrical cavity into three distinct compartments. The disks are assumed to be impermeable, and fitting snugly, they in general permit no movement of gas across their plane. However, they also are assumed to be able to move without any friction.

Initially, all the three compartments are filled with a compressible fluid to the same pressure in each compartment, say 1 atm. Since all the three compartments are at the same pressure, the disks stay stationary.

Then, suppose that the piston on the extreme right end is moved, say from position P_1 to P_2. The final position P_2 may be to the left or to the right of the initial position P_1; it doesn’t matter. For the current description, however, let’s suppose that the position P_2 is to the left of P_1. The effect of the piston movement thus is to increase the pressure inside the system.

The problem is to determine the nature of the resulting displacements that the two disks undergo as measured from their respective initial positions.

There are essentially two entirely different paradigms for conducting an analysis of this problem.


The “Vector Mechanics” Paradigm:

The first paradigm is based on an approach that was put to use so successfully by Newton. Usually, it is called the paradigm of vector analysis.

In this paradigm, we focus on the fact that the forced displacement of the piston with time, x(t), may be described using some function of time that is defined over the interval lying between two instants t_i and t_f.

For example, suppose the function is:
x(t) = x_0 + v t,
where v is a constant. In other words, the motion of the piston is steady, with a constant velocity, between the initial and final instants. Since the velocity is constant, there is no acceleration over the open interval (t_i, t_f).

However, notice that before the instant t_i, the piston velocity was zero. Then, the velocity suddenly became a finite (constant) value. Therefore, if you extend the interval to include the end-instants as well, i.e., if you consider the semi-closed interval [t_i, t_f), then there is an acceleration at the instant t_i. Similarly, since the piston comes to a position of rest at t = t_f, there also is another acceleration, equal in magnitude and opposite in direction, which appears at the instant t_f.

The existence of these two instantaneous accelerations implies that jerks or pressure waves are sent through the system. We may model them as vector quantities, as impulses. [Side Exercise: Work out what happens if we consider only the open interval (t_i, t_f).]

We can now apply Newton’s 3 laws, based on the idea that shock-waves must have begun at the piston at the instant t = t_i. They must have got transmitted through the gas kept under pressure, and they must have affected the disk D_1 lying closest to the piston, thereby setting this disk into motion. This motion must have passed through the gas in the middle compartment of the system as another pulse in the pressure (generated at the disk D_1), thereby setting also the disk D_2 in a state of motion a little while later. Finally, the pulse must have got bounced off the seal on the left hand side, and in turn, come back to affect the motion of the disk D_2, and then of the disk D_1. Continuing their travels to and fro, the pulses, and hence the disks, would thus be put in a back and forth motion.

After a while, these transients would move forth and back, superpose, and some of their constituent frequencies would get cancelled out, leaving only those frequencies operative such that the three compartments are put under some kind of stationary states.

In case the gas is not ideal, there would be damping anyway, and after a sufficiently long while, the disks would move through such small displacements that we could easily ignore the ever-decreasing displacements in a limiting argument.

Thus, assume that, after an elapse of a sufficiently long time, the disks become stationary. Of course, their new positions are not the same as their original positions.

The problem thus can be modeled as basically a transient one. The state of the new equilibrium state is thus primarily seen as an effect or an end-result of a couple of transient processes which occur in the forward and backward directions. The equilibrium is seen as not a primarily existing state, but as a result of two equal and opposite transient causes.

Notice that throughout this process, Newton’s laws can be applied directly. The nature of the analysis is such that the quantities in question—viz. the displacements of the disks—always are real, i.e., they correspond to what actually is supposed to exist in the reality out there.

The (values of) displacements are real in the sense that the mathematical analysis procedure itself involves only those (values of) displacements which can actually occur in reality. The analysis does not concern itself with some other displacements that might have been possible but don’t actually occur. The analysis begins with the forced displacement condition, translates it into pressure waves, which in turn are used in order to derive the predicted displacements in the gas in the system, at each instant. Thus, at any arbitrary instant of time t > t_i (in fact, the analysis here runs for times t \gg t_f), the analysis remains concerned only with those displacements that are actually taking place at that instant.

The Method of Calculus of Variations:

The second paradigm follows the energetics program. This program was initiated by Newton himself as well as by Leibnitz. However, it was pursued vigorously not by Newton but rather by Leibnitz, and then by a series of gifted mathematicians-physicists: the Bernoulli brothers, Euler, Lagrange, Hamilton, and others. This paradigm is essentially based on the calculus of variations. The idea here is something like the following.

We do not care for a local description at all. Thus, we do not analyze the situation in terms of the local pressure pulses, their momenta/forces, etc. All that we focus on are just two sets of quantities: the initial positions of the disks, and their final positions.

For instance, focus on the disk D_1. It initially is at the position x_{1_i}. It is found, after a long elapse of time (i.e., at the next equilibrium state), to have moved to x_{1_f}. The question is: how to relate this change in x_1 on the one hand, to the displacement that the piston itself undergoes from P_{x_i} to P_{x_f}.

To analyze this question, the energetics program (i.e., the calculus of variations) adopts a seemingly strange methodology.

It begins by saying that there is nothing unique to the specific value of the position x_{1_f} as assumed by the disk D_1. The disk could have come to a halt at any other (nearby) position, e.g., at some other point x_{1_1}, or x_{1_2}, or x_{1_3}, … etc. In fact, since there are an infinity of points lying in a finite segment of line, there could have been an infinity of positions where the disk could have come to a rest, when the new equilibrium was reached.

Of course, in reality, the disk D_1 comes to a halt at none of these other positions; it comes to a halt only at x_{1_f}.

Yet, the theory says, we need to be “all-inclusive,” in a way. We need not, just for the aforementioned reason, deny a place in our analysis to these other positions. The analysis must include all such possible positions—even if they be purely hypothetical, imaginary, or unreal. What we do in the analysis, this paradigm says, is to initially include these merely hypothetical, unrealistic positions too on exactly the same footing as that enjoyed by that one position which is realistic, which is given by x_{1_f}.

Thus, we take a set of all possible positions for each disk. Then, for each such a position, we calculate the “impact” it would make on the energy of the system taken as a whole.

The energy of the system can be additively decomposed into the energies carried by each of its sub-parts. Thus, focusing on disk D_1, for each one of its possible (hypothetical) final position, we should calculate the energies carried by both its adjacent compartments. Since a change in D_1‘s position does not affect the compartment 3, we need not include it. However, for the disk D_1, we do need to include the energies carried by both the compartments 1 and 2. Similarly, for each of the possible positions occupied by the disk D_2, it should include the energies of the compartments 2 and 3, but not of 1.

At this point, to bring simplicity (and thereby better) clarity to this entire procedure, let us further assume that the possible positions of each disk forms a finite set. For instance, each disk can occupy only one of the positions that is some -5, -4, -3, -2, -1, 0, +1, +2, +3, +4 or +5 distance-units away from its initial position. Thus, a disk is not allowed to come to a rest at, say, 2.3 units; it must do so either at 2 or at 3 units. (We will thus perform the initial analysis in terms of only the integer positions, and only later on extend it to any real-valued positions.) (If you are a mechanical engineering student, suggest a suitable mechanism that can ensure only integer relative displacements.)

The change in energy E of a compartment is given by
\Delta E = P A \Delta x,
where P is the pressure, A is the cross-sectional area of the cylinder, and \Delta x is the change in the length of the compartment.

Now, observe that the energy of the middle compartment depends on the relative distance between the two disks lying on its sides. Yet, for the same reason, the energy of the middle compartment does depend on both these positions. Hence, we must take a Cartesian product of the relative displacements undergone by both the disks, and only then calculate the system energy for each such a permutation (i.e. the ordered pair) of their positions. Let us go over the details of the Cartesian product.

The Cartesian product of the two positions may be stated as a row-by-row listing of ordered pairs of the relative positions of D_1 and D_2, e.g., as follows: the ordered pair (-5, +2) means that the disk D_1 is 5 units to the left of its initial position, and the disk D_2 is +2 units to the right of its initial position. Since each of the two positions forming an ordered pair can range over any of the above-mentioned 11 number of different values, there are, in all, 11 \times 11 = 121 number of such possible ordered pairs in the Cartesian product.

For each one of these 121 different pairs, we use the above-given formula to determine what the energy of each compartment is like. Then, we add the three energies (of the three compartments) together to get the value of the energy of the system as a whole.

In short, we get a set of 121 possible values for the energy of the system.

You must have noticed that we have admitted every possible permutation into analysis—all the 121 number of them.

Of course, out of all these 121 number of permutations of positions, it should turn out that 120 number of them have to be discarded because they would be merely hypothetical, i.e. unreal. That, in turn, is because, the relative positions of the disks contained in one and only one ordered pair would actually correspond to the final, equilibrium position. After all, if you conduct this experiment in reality, you would always get a very definite pair of the disk-positions, and it this same pair of relative positions that would be observed every time you conducted the experiment (for the same piston displacement). Real experiments are reproducible, and give rise to the same, unique result. (Even if the system were to be probabilistic, it would have to give rise to an exactly identical probability distribution function.) It can’t be this result today and that result tomorrow, or this result in this lab and that result in some other lab. That simply isn’t science.

Thus, out of all those 121 different ordered-pairs, one and only one ordered-pair would actually correspond to reality; the rest all would be merely hypothetical.

The question now is, which particular pair corresponds to reality, and which ones are unreal. How to tell the real from the unreal. That is the question.

Here, the variational principle says that the pair of relative positions that actually occurs in reality carries a certain definite, distinguishing attribute.

The system-energy calculated for this pair (of relative displacements) happens to carry the lowest magnitude from among all possible 121 number of pairs. In other words, any hypothetical or unreal pair has a higher amount of system energy associated with it. (If two pairs give rise to the same lowest value, both would be equally likely to occur. However, that is not what provably happens in the current example, so let us leave this kind of a “degeneracy” aside for the purposes of this post.)

(The update on 26 December 2016 begins here:)

Actually, the description  given in the immediately preceding paragraph was a bit too dumbed down. The variational principle is more subtle than that. Explaining it makes this post even longer, but let me give it a shot anyway, at least today.

To follow the actual idea of the variational principle (in a not dumbed-down manner), the procedure you have to follow is this.

First, make a table of all possible relative-position pairs, and their associated energies. The table has the following columns: a relative-position pair, the associated energy E as calculated above, and one more column which for the time being would be empty. The table may look something like what the following (partial) listing shows:

(0,0) -> say, 115 Joules
(-1,0) -> say, 101 Joules
(-2,0) -> say, 110 Joules

(2,2) -> say, 102 Joules
(2,3) -> say, 100 Joules
(2,4) -> say, 101 Joules
(2,5) -> say, 120 Joules

(5,0) -> say, 135 Joules

(5,5) -> say 117 Joules.

Having created this table (of 121 rows), you then pick each row one by and one, and for the picked up n-th row, you ask a question: What all other row(s) from this table have their relative distance pairs such that these pairs lie closest to the relative distance pair of this given row. Let me illustrate this question with a concrete example. Consider the row which has the relative-distance pair given as (2,3). Then, the relative distance pairs closest to this one would be obtained by adding or subtracting a distance of 1 to each in the pair. Thus, the relative distance pairs closest to this one would be: (3,3), (1,3), (2,4), and (2,2). So, you have to pick up those rows which have these four entries in the relative-distance pairs column. Each of these four pairs represents a variation \delta on the chosen state, viz. the state (2,3).

In symbolic terms, suppose for the n-th row being considered, the rows closest to it in terms of the differences in their relative distance pairs, are the a-th, b-th, c-th and d-th rows. (Notice that the rows which are closest to a given row in this sense, would not necessarily be found listed just above or below that given row, because the scheme followed while creating the list or the vector that is the table would not necessarily honor the closest-lying criterion (which necessarily involves two numbers)—not at least for all rows in the table.

OK. Then, in the next step, you find the differences in the energies of the n-th row from each of these closest rows, viz., the a-th, b-th, c-th and c-th rows. That is to say, you find the absolute magnitudes of the energy differences. Let us denote these magnitudes as: \delta E_{na} = |E_n - E_a|\delta E_{nb} = |E_n - E_b|\delta E_{nc} = |E_n - E_c| and \delta E_{nd} = |E_n - E_d|.  Suppose the minimum among these values is \delta E_{nc}. So, against the n-th row, in the last column of the table, you write the value \delta E_{nc}.

Having done this exercise separately for each row in the table, you then ask: Which row has the smallest entry in the last column (the one for \delta E), and you pick that up. That is the distinguished (or the physically occurring) state.

In other words, the variational principle asks you to select not the row with the lowest absolute value of energy, but that row which shows the smallest difference of energy from one of its closest neighbours—and these closest neighbours are to be selected according to the differences in each number appearing in the relative-distance pair, and not according to the vertical place of rows in the tabular listing. (It so turns out that in this example, the row thus selected following both criteria—lowest energy as well as lowest variation in energy—are identical, though it would not necessarily always be the case. In short, we can’t always get away with the first, too dumbed down, version.)

Thus, the variational principle is about that change in the relative positions for which the corresponding change in the energy vanishes (or has the minimum possible absolute magnitude, in case the positions form a discretely varying, finite set).

(The update on 26th December 2016 gets over here.)

And, it turns out that this approach, too, is indeed able to perfectly predict the final disk-positions—precisely as they actually are observed in reality.

If you allow a continuum of positions (instead of the discrete set of only the 11 number of different final positions for one disk, or 121 number of ordered pairs), then instead of taking a Cartesian product of positions, what you have to do is take into account a tensor product of the position functions. The maths involved is a little more advanced, but the underlying algebraic structure—and the predictive principle which is fundamentally involved in the procedure—remains essentially the same. This principle—the variational principle—says:

Among all possible variations in the system configurations, that system configuration corresponds to reality which has the least variation in energy associated with it.

(This is a very rough statement, but it will do for this post and for a general audience. In particular, we don’t look into the issues of what constitute the kinematically admissible constraints, why the configurations must satisfy the field boundary conditions, the idea of the stationarity vs. of a minimum or a maximum, i.e., the issue of convexity-vs.-concavity, etc. The purpose of this post—and our example here—are both simple enough that we need not get into the whole she-bang of the variational theory as such.)

Notice that in this second paradigm, (i) we did not restrict the analysis to only those quantities that are actually taking place in reality; we also included a host (possibly an infinity) of purely hypothetical combinations of quantities too; (ii) we worked with energy, a scalar quantity, rather than with momentum, a vector quantity; and finally, (iii) in the variational method, we didn’t bother about the local details. We took into account the displacements of the disks, but not any displacement at any other point, say in the gas. We did not look into presence or absence of a pulse at one point in the gas as contrasted from any other point in it. In short, we did not discuss the details local to the system either in space or in time. We did not follow the system evolution, at all—not at least in a detailed, local way. If we were to do that, we would be concerned about what happens in the system at the instants and at spatial points other than the initial and final disk positions. Instead, we looked only at a global property—viz. the energy—whether at the sub-system level of the individual compartments, or at the level of the overall system.


The Two Paradigms Contrasted from Each Other:

If we were to follow Newton’s method, it would be impossible—impossible in principle—to be able to predict the final disk positions unless all their motions over all the intermediate transient dynamics (occurring over each moment of time and at each place of the system) were not be traced. Newton’s (or vectorial) method would require us to follow all the details of the entire evolution of all parts of the system at each point on its evolution path. In the variational approach, the latter is not of any primary concern.

Yet, in following the energetics program, we are able to predict the final disk positions. We are able to do that without worrying about what all happened before the equilibrium gets established. We remain concerned only with certain global quantities (here, system-energy) at each of the hypothetical positions.

The upside of the energetics program, as just noted, is that we don’t have to look into every detail at every stage of the entire transient dynamics.

Its downside is that we are able to talk only of the differences between certain isolated (hypothetical) configurations or states. The formalism is unable to say anything at all about any of the intermediate states—even if these do actually occur in reality. This is a very, very important point to keep in mind.


The Question:

Now, the question with which we began this post. Namely, what does the delta of the variational calculus mean?

Referring to the above discussion, note that the delta of the variational calculus is, here, nothing but a change in the position-pair, and also the corresponding change in the energy.

Thus, in the above example, the difference of the state (2,3) from the other close states such as (3,3), (1,3), (2,4), and (2,2) represents a variation in the system configuration (or state), and for each such a variation in the system configuration (or state), there is a corresponding variation in the energy \delta E_{ni} of the system. That is what the delta refers to, in this example.

Now, with all this discussion and clarification, would it be possible for you to clearly state what the physical meaning of the delta is? To what precisely does the concept refer? How does the variation in energy \delta E differ from both the finite changes (\Delta E) as well as the infinitesimal changes (\text{d}E) of the usual calculus?


Note, the question is conceptual in nature. And, no, not a single one of the very best books on classical mechanics manages to give a very succinct and accurate answer to it. Not even Rana and Joag (or Goldstein, or Feynman, or…)

I will give my answer in my next post, next year. I will also try to apply it to a couple of more interesting (and somewhat more complicated) physical situations—one from engineering sciences, and another from quantum mechanics!

In the meanwhile, think about it—the delta—the concept itself, its (conceptual) meaning. (If you already know the calculus of variations, note that in my above write-up, I have already supplied the answer, in a way. You just have to think a bit about it, that’s all!)


An Important Note: Do bring this post to the notice of the Officially Approved Full Professors of Mechanical Engineering in SPPU, and the SPPU authorities. I would like to know if the former would be able to state the meaning—at least now that I have already given the necessary context in such great detail.

Ditto, to the Officially Approved Full Professors of Mechanical Engineering at COEP, esp. D. W. Pande, and others like them.

After all, this topic—Lagrangian mechanics—is at the core of Mechanical Engineering, even they would agree. In fact, it comes from a subject that is not taught to the metallurgical engineers, viz., the topic of Theory of Machines. But it is taught to the Mechanical Engineers. That’s why, they should be able to crack it, in no time.

(Let me continue to be honest. I do not expect them to be able to crack it. But I do wish to know if they are able at least to give a try that is good enough!)


Even though I am jobless (and also nearly bank balance-less, and also cashless), what the hell! …

…Season’s greetings and best wishes for a happy new year!


A Song I Like:

[With jobless-ness and all, my mood isn’t likely to stay this upbeat, but anyway, while it lasts, listen to this song… And, yes, this song is like, it’s like, slightly more than 60 years old!]

(Hindi) “yeh raat bhigee bhigee”
Music: Shankar-Jaikishan
Singers: Manna De and Lata Mangeshkar
Lyrics: Shailendra


[E&OE]