Machine “Learning”—An Entertainment [Industry] Edition

Yes, “Machine ‘Learning’,” too, has been one of my “research” interests for some time by now. … Machine learning, esp. ANN (Artificial Neural Networks), esp. Deep Learning. …

Yesterday, I wrote a comment about it at iMechanica. Though it was made in a certain technical context, today I thought that the comment could, perhaps, make sense to many of my general readers, too, if I supply a bit of context to it. So, let me report it here (after a bit of editing). But before coming to my comment, let me first give you the context in which it was made:


Context for my iMechanica comment:

It all began with a fellow iMechanician, one Mingchuan Wang, writing a post of the title “Is machine learning a research priority now in mechanics?” at iMechanica [^]. Biswajit Banerjee responded by pointing out that

“Machine learning includes a large set of techniques that can be summarized as curve fitting in high dimensional spaces. [snip] The usefulness of the new techniques [in machine learning] should not be underestimated.” [Emphasis mine.]

Then Biswajit had pointed out an arXiv paper [^] in which machine learning was reported as having produced some good DFT-like simulations for quantum mechanical simulations, too.

A word about DFT for those who (still) don’t know about it:

DFT, i.e. Density Functional Theory, is “formally exact description of a many-body quantum system through the density alone. In practice, approximations are necessary” [^]. DFT thus is a computational technique; it is used for simulating the electronic structure in quantum mechanical systems involving several hundreds of electrons (i.e. hundreds of atoms). Here is the obligatory link to the Wiki [^], though a better introduction perhaps appears here [(.PDF) ^]. Here is a StackExchange on its limitations [^].

Trivia: Kohn and Sham received a Physics Nobel for inventing DFT. It was a very, very rare instance of a Physics Nobel being awarded for an invention—not a discovery. But the Nobel committee, once again, turned out to have put old Nobel’s money in the right place. Even if the work itself was only an invention, it did directly led to a lot of discoveries in condensed matter physics! That was because DFT was fast—it was fast enough that it could bring the physics of the larger quantum systems within the scope of (any) study at all!

And now, it seems, Machine Learning has advanced enough to be able to produce results that are similar to DFT, but without using any QM theory at all! The computer does have to “learn” its “art” (i.e. “skill”), but it does so from the results of previous DFT-based simulations, not from the theory at the base of DFT. But once the computer does that—“learning”—and the paper shows that it is possible for computer to do that—it is able to compute very similar-looking simulations much, much faster than even the rather fast technique of DFT itself.

OK. Context over. Now here in the next section is my yesterday’s comment at iMechanica. (Also note that the previous exchange on this thread at iMechanica had occurred almost a year ago.) Since it has been edited quite a bit, I will not format it using a quotation block.


[An edited version of my comment begins]

A very late comment, but still, just because something struck me only this late… May as well share it….

I think that, as Biswajit points out, it’s a question of matching a technique to an application area where it is likely to be of “good enough” a fit.

I mean to say, consider fluid dynamics, and contrast it to QM.

In (C)FD, the nonlinearity present in the advective term is a major headache. As far as I can gather, this nonlinearity has all but been “proved” as the basic cause behind the phenomenon of turbulence. If so, using machine learning in CFD would be, by the simple-minded “analysis”, a basically hopeless endeavour. The very idea of using a potential presupposes differential linearity. Therefore, machine learning may be thought as viable in computational Quantum Mechanics (viz. DFT), but not in the more mundane, classical mechanical, CFD.

But then, consider the role of the BCs and the ICs in any simulation. It is true that if you don’t handle nonlinearities right, then as the simulation time progresses, errors are soon enough going to multiply (sort of), and lead to a blowup—or at least a dramatic departure from a realistic simulation.

But then, also notice that there still is some small but nonzero interval of time which has to pass before a really bad amplification of the errors actually begins to occur. Now what if a new “BC-IC” gets imposed right within that time-interval—the one which does show “good enough” an accuracy? In this case, you can expect the simulation to remain “sufficiently” realistic-looking for a long, very long time!

Something like that seems to have been the line of thought implicit in the results reported by this paper: [(.PDF) ^].

Machine learning seems to work even in CFD, because in an interactive session, a new “modified BC-IC” is every now and then is manually being introduced by none other than the end-user himself! And, the location of the modification is precisely the region from where the flow in the rest of the domain would get most dominantly affected during the subsequent, small, time evolution.

It’s somewhat like an electron rushing through a cloud chamber. By the uncertainty principle, the electron “path” sure begins to get hazy immediately after it is “measured” (i.e. absorbed and re-emitted) by a vapor molecule at a definite point in space. The uncertainty in the position grows quite rapidly. However, what actually happens in a cloud chamber is that, before this cone of haziness becomes too big, comes along another vapor molecule, and “zaps” i.e. “measures” the electron back on to a classical position. … After a rapid succession of such going-hazy-getting-zapped process, the end result turns out to be a very, very classical-looking (line-like) path—as if the electron always were only a particle, never a wave.

Conclusion? Be realistic about how smart the “dumb” “curve-fitting” involved in machine learning can at all get. Yet, at the same time, also remain open to all the application areas where it can be made it work—even including those areas where, “intuitively”, you wouldn’t expect it to have any chance to work!

[An edited version of my comment is over. Original here at iMechanica [^]]


 

“Boy, we seem to have covered a lot of STEM territory here… Mechanics, DFT, QM, CFD, nonlinearity. … But where is either the entertainment or the industry you had promised us in the title?”

You might be saying that….

Well, the CFD paper I cited above was about the entertainment industry. It was, in particular, about the computer games industry. Go check out SoHyeon Jeong’s Web site for more cool videos and graphics [^], all using machine learning.


And, here is another instance connected with entertainment, even though now I am going to make it (mostly) explanation-free.

Check out the following piece of art—a watercolor landscape of a monsoon-time but placid sea-side, in fact. Let me just say that a certain famous artist produced it; in any case, the style is plain unmistakable. … Can you name the artist simply by looking at it? See the picture below:

A sea beach in the monsoons. Watercolor.

If you are unable to name the artist, then check out this story here [^], and a previous story here [^].


A Song I Like:

And finally, to those who have always loved Beatles’ songs…

Here is one song which, I am sure, most of you had never heard before. In any case, it came to be distributed only recently. When and where was it recorded? For both the song and its recording details, check out this site: [^]. Here is another story about it: [^]. And, if you liked what you read (and heard), here is some more stuff of the same kind [^].


Endgame:

I am of the Opinion that 99% of the “modern” “artists” and “music composers” ought to be replaced by computers/robots/machines. Whaddya think?

[Credits: “Endgame” used to be the way Mukul Sharma would end his weekly Mindsport column in the yesteryears’ Sunday Times of India. (The column perhaps also used to appear in The Illustrated Weekly of India before ToI began running it; at least I have a vague recollection of something of that sort, though can’t be quite sure. … I would be a school-boy back then, when the Weekly perhaps ran it.)]

 

Micro-level water-resources engineering

1. Introductory:

It’s the “Holi” day today—one of the two Indian cultural reminders that the summer is almost at the door-step. [The second reminder is the “Gudhi PaaDawaa,” the Indian lunar new year’s day, which follows after a lunar fortnight.]

The hottest—and the most water-scarce—days of late-April, May and early-June are in the coming.

This period also is the last opportunity in the year to undertake any appropriate water-conservation work.

Those who have browsed my personal Website would know that surface-flow and ground-water seepage has been a topic of some definite research interest to me. [Off-hand, I think I also have passingly mentioned about it on my blog in the past.] … That way, I haven’t actually pursued any concrete research about it; it’s just been in an exploratory stage. I do plan to do something about it once I get some right kind of students to guide along these topics, preferably ME students (from several different disciplines; see my research description at the end of this post). [BTW, even though currently I am jobless, I do anticipate to get a job in the next cycle of academic appointments that occurs sometime around the summer vacations or so.]

In the meanwhile, I have been going over some popular as well as scientific writings on the subject, thinking over the issues involved, and bringing some clarity as to what in particular I can do about it. My research would involve only computational modelling. In particular, I wouldn’t at all be interested in the sociological/governmental aspects of it, though one must be aware that they exist, and one must have at least some background kind of a sense of what they are like.

There has been a lot of coverage in the media about some of these initiatives/work. Three stand out, in the chronological order: (i) Rajendra Singh’s work in Rajasthan, (ii) Anna Hazaare’s, in Ralegan Siddhi in Maharashtra, and (iii) Suresh Khanapurkar and Amrishbhai Patel’s, in Shirpur, Maharashtra.  As usual, media’s coverage of these efforts is mostly superficial, partial/incomplete, and skewed.

Here are some of my notes after browsing about these three efforts.

1. Rajendra Singh’s work:

Rajendra Singh has by now become something of a celebrity among the NGO-type of social workers; today he even attracts the wide-eyed young (and mostly clueless) volunteers from the urban areas.

I would strongly suggest you to pursue your own browsing about Singh’s work, starting, e.g., here [^], before reading further.

Please do that first, in order to realize the extraordinary perceptiveness of this piece, written by Amanda Suutari and Gerry Marten, here [^]. I don’t know who the authors are, except for their profiles here [^]. The main article at both the links is the same.

IMO, this piece is the best among all the articles available on this topic on the Internet. Yes, this piece, too, has some pinkish shades at places. “Commercial mines” is far wider a term that seems to have deliberately been put to use here; the mines actually were relatively small, shallow, and only for the marble stone, not for other minerals. That said, still, the aforementioned pink is rather rare in that article, and it occurs mostly in some minor places that are fairly well isolated. I mean, the entirety of the article itself has not been deliberately painted with a background pinkish wash of sorts. And if you go through the article ignoring these isolated streaks of the pink, then there is a wealth of accurate observations, and minute but relevant detail. The article truly stands out from the crowd.

As far as Singh’s main work goes, from an engineer’s point of view, here are some of my unanswered questions or points:

The remote rural area of Rajasthan that Singh famously went to (and stayed in) is not in the Thar desert, but in the Aravali mountains. From the Google Earth perspective, this location is just about a stone’s throw from Jaipur—and also from the then still surviving forest park. The “before” photos, too, show some greenery on some hills—not those seemingly endless yellow and wavy sand dunes flatly spreading everywhere up to the horizon. Why must every media report emphasize “Rajasthan” as a whole, when they talk about Singh’s work? Why don’t they say the half-green Aravali parts near Delhi and MP? Two further sub-points:

  • How effective would the collection of the falling rain be, if the region weren’t to be mountainous?
  • To what extent does the geological structure of the mountains and the flatter land help make Singh’s approach successful? The upper layers there are alluvial.

The usual criterion of repeatability or replicability: Singh did achieve repeated success in other villages too. In fact, in hundreds of other villages—800+ villages, in fact! Good! No, Great!!

Still, notice, all these villages lie in the same region—a comparatively very small part (area-wise certainly less than 5%) of the entire state of Rajasthan. Did Singh, especially after the Magsaysay award (2001), try something about 400 km west? at least about 200 km west? Why not?

Why does the media still insist on saying that it was a success in the arid lands of Rajasthan—as if all the representative parts of Rajasthan had been successfully demonstrated to benefit from the scheme?

In summary: Singh did demonstrate the very feasibility of this micro-level approach, going against the then existing engineering wisdom. Congratulations! Singh also did replicate his initial success at hundreds of other locations in Rajasthan—though not at a majority of places—or even a representative minority of places—in that state. Our optimism should be guarded. [Also, though I didn’t mention it, observe the government tried to spoil Singh’s work. When governments enter the economy, they are like that, regardless of who peoples the government.]

2. Anna Hazaare’s “Work” in Ralegan Siddhi:

Ah, Anna Hazaare! … I have written about this fellow before. Regardless of that, let me say, it’s impossible to hold a lasting grudge against this guy. The main reason is that one doesn’t hold grudges—there is no need to do that if you are willing to pass your moral judgements. The other reason is supplied by his personality: his appearance, mannerisms, language, “thoughts,” actions (remember him running after breaking his fast in Delhi?)… All such things included. …

… Hazaare’s is a personality of a very exceptional kind: he is a walking & talking, breathing & living, caricature. And he also is very forceful about what he does. … A forceful 3D living caricature that is busy building castles in the thin air at all times. How would it even be possible to take him seriously? Not unless the media makes an elephant out of him, and then insists on using the TV to make it sit in the room—your living room.

But let’s keep that aside, and let’s try to look at his water-conservation “work” in Ralegan Siddhi. How successful has the effort been, given its geographical and other contexts? A few notes of mine follow:

What is the extent of the greening that has been effected in Ralegan Siddhi? How does it compare (keeping all other factors equal or comparable) to the average greenery within an area of 50 km radius? Or even the other drought-prone region right in the same district? Answer: not at all impressive—if you are an honest observer, that is.

If Hazaare were not to arrange to divert water from the conventional irrigation canal running nearby to Ralegan Siddhi, if he were to rely only on his local, micro-level, water conservation schemes, how successful could he have been? About 25–30%, at the most, of what you presently see there, some engineers estimate. If he now were to agree not to take any water from the nearby Kukadi project canal, how long would it take for the existing Ralegan Siddhi greenery to turn yellow/brown? My estimate, after discussions with some engineers: about 5 to 10 years, with the lower side being much more likely. Note, this is a period far shorter than the one for which Hazaare has been continuously lauded in the media (and in the successive state governments) for his water-conservation “work.”

Replication: The Maharashtra government has wasted 100+ crores on this “Gandhian”‘s hopeless dreams. Why couldn’t they achieve success anywhere else—not at a single site elsewhere? Hazaare’s and media’s answer: It’s all Maharashtra government’s fault. (LOL!)

Note, Singh did succeed in hundreds of other villages—initially (and for a long time), without taking a single penny from the government funds. Hazaare did not succeed in a single other village, despite hundred+ of crores.

Summary: Idiocy, hypocracy, and media hype. Plus, shameless loot of the credit actually due to the conventional irrigation engineering.

3. Suresh Khanapurkar and Amrishbhai Patel’s work in Shirpur:

OK. With sections 1. and 2., we are already done with the notable works done in the 20th century. Both were (or at least have been called) “Gandhian.” Now, we enter the 21st century, and the matters do get a bit more more complicated—also, better funded, better documented, and on the whole, more interesting, anyway.

Summary: Khanapurkar is a geologist, and has retired from a government job. He has been an RSS guy. Amrishbhai Patel always has been an Indira Congress guy, an MLA too. But, he is a Patel. [Aakar?] As to their work: as (almost) always (at least in Maharashtra), when it comes to some secular/non-religious kind of a social work, first, someone from the Congress leads the way; if successful, The Family is given the entire credit; then, the “jholawaalaa”s eagerly follow; then some RSS guy enters the scene and attempts some improvement on the original theme, which often is unsuccessful, or at least, it is not just as successful; then the pinkos use the RSS guy’s failure to attack the RSS; then the RSS/RSS guy make(s) deal with the government/local powers; around this time, the RSS recedes into the background and the RSS guy finally begins to shine in the limelight; then more funds follow; then some more critical “jholawaalaa”s follow, and, simultaneously, the other pinkos and the reds wait and watch.

With the pressure of providing a very short and succinct summary being out of the way, we may now look at the situation from the engineer’s perspective.

While covering Hazaare’s “work,” I did not care to provide any link. The resources are over-abundant, and, as expected, none covers the ground reality the way it should be. For example, none discusses the extent of contribution of the Kukadi project canal; none mentions the hundred+ crores already wasted by the successive Maharashtra governments on Anna’s day-dreams “thoughts.”

In contrast, for the Shirpur pattern, there is an objective need to provide links. Reasons:

  • The Shirpur pattern has been tried elsewhere with some success, e.g., at the initiative of the NCP in the drought-prone areas in southern Maharashtra.
  • There is a BJP government in the Center, and a BJP-led government in the State.
  • The new BJP budget at the Center has announced thousands of crores for micro-level water-resources management: Rs. 5,300 crores nationwide, i.e., about 850 Million US dollars—say, almost a billion dollars.
  • The Shirpur pattern is open to a critical scrutiny, and not just of the same kind as Singh’s work invites, viz., the relevance of the geological factors, and the feasibility (perhaps with local adaptations/changes) or otherwise of replication. In addition to those two factors, the Shirpur pattern also remains open to an additional serious criticism, one concerning the undesirable and highly under-appreciated side-effects. And, this point acquires urgency because of the first three points.

Hence for the Shirpur pattern, I sure wish to provide at least some links. These follow, with a few notes of mine:

Here is a typical introductory sort of an article on this topic that would appear before the state/central governments began supporting the idea: [^]. I anticipate that much better written (and better-formatted) articles would arrive in the near future.

The model seems to work also elsewhere: [^].

Again, a perceptive piece, despite the fact that it seems to come from someone with pinkish inclinations: [^]. The author for the preceding piece is one K. J. Joy [^]. His name means that he must be at least a pink if not a red. [Aakaar?] He is something of that sort! “Privatization can do more harm than good” [^]. OK. Humour apart, even if his understanding of the terms such as “rights” (i.e., more properly, “individual rights”) and “privatization” does not seem to be sufficiently clear, it still does not mean that his article itself isn’t studious or valuable. Do go through the article; highly recommended.

A well-informed criticism; note especially the important and relevant geological points: [^]

An article that cites some actual geological data. Though the data are far too coarse-grained to be of any direct use in any micro-scale schemes, the article at least cares to look into some factual data. … You are not surprised by the author’s background, are you? [^]

An indication of the kind of complexity there is, in implementation: [^]

An example of the usual “our region didn’t get its share” [^]; such things seem to have begun already! Note, the demand has been made without pausing to think anything about whether or how the approach might at all work in a given area. I am not saying that the approach wouldn’t work in Marathwada—another region of severe droughts. In fact one of the links I gave above already indicate some success for this approach in that region too. Here, I am just highlighting the kind of artificial tensions that come in whenever governments interfere with the economy. And, I am saying, without being cynical about anything: “more research is necessary.”

4. A word about my planned research:

Here is an outline of the way my planned research might go:

  • Initially, (i) build a computer model of the surface topology and the underground geological strata and structures for some area—this could even be an imaginary geographical area!; and then, (ii) develop/adapt algorithms to simulate groundwater seepage after precipitation in this area, running the simulation for, may be, a decade or so. The quantities for the precipitation and the surface flow would enter the model simply as assumed boundary data, that’s all.
  • Add features to incorporate small check-dams or other structures at various locations and scales, and study their effect on groundwater seepage and water-table levels.
  • Add the features of the surface water flow and study aspects such as flooding vs. seepage, etc.
  • Then, take a focus area—an actually existing drought-prone area—and study its precipitation and geological features, build models, run simulations, and make some recommendations for locations of check dams and other structures/features.

The above is a broad conceptual outline that I currently have in mind. In the actual research, some components of some of the steps may get mixed up, and some other steps may get added in, e.g., a step of: simulating the effect on groundwater seepage and water-table levels, due to closure of an aquifer that got exposed due to digging of deep trenches while implementing the Shirpur pattern.

The research should actually begin after I land a professor’s job. In the meanwhile, enthusiastic engineers with programming knowledge may feel free to approach me—but only if they are willing to work hard, and for free! … When I play, I play, but when I work, I really work. Usually, that means hard work, at least compared to many, many others. So, don’t approach me unless you already know what it takes to do hard work over a considerably long period of time—at least months. (As far as I know, no smart work ever comes before at least a certain quantum of some very hard work has gone before it.) Also, I have no money to support you—or, for that matter, as of today, even myself! But if it still is all OK by you, and you still wish to do something in this direction working with me, then do feel free to drop me a line. Use email or comment form (and feel free to mention that you want to keep the comment confidential—comments here are moderated). I am serious about this stuff.

* * * * *   * * * * *   * * * * *

A Song I Like:
(Hindi) “yeh shaam mastaani…”
Singer: Kishore Kumar
Music: R. D. Burman
Lyrics: Anand Bakshi

[I guess the post already is in a fairly good shape. I would update it only if I find some more interesting links etc.; otherwise, I would leave it alone as is. I mean, adding updates for streamlining and clarifying are much less likely here. Anyway, bye for now… ]

[E&OE]

.

So, it is a QC (at least this week!)

I wanted to write on tensors etc., but a few very fresh inputs concerning the D-Wave device have appeared, all barely within the past 24 hours or less.

First, it was Prof. David Poulin commenting at Prof. Scott Aaronson’s blog once again [^], alerting some new work from Prof. Troyer. Unlike in his last comment (on the same post, when he thought that it was not a QC), Poulin has now come closer towards (or has started) supporting the position that the D-Wave device is a QC:

“…the problem instances that are easy for the D-wave device can sometimes be hard for the SS model. This is interesting new evidence supporting the quantum nature of the D-wave device.”

Next, a very valuable comment by one Bill Kaminsky appeared on Aaronson’s blog, very neatly explaining the Smolin and Smith model [^], and then contrasting it with the new result by Troyer. [Guess this Bill Kaminsky is the same as one William Kaminsky, who, in turn, is a PhD student in QIS at MIT. (… Just a Google search, that’s all!)] … Incidentally, more explanatory material concerning the adiabatic quantum optimization, quantum annealing, and classical annealing, written by Kaminsky, had already been put up last week at Henning Dekant’s blog; see here [^].

Finally, while idly thinking about all these things, even as idly browsing Prof. Poulin’s home page, I just idly happened to hit the “New on quant-ph” link [^] at its bottom, and thereby landed at the arXiv site; and once there, I noticed a new paper by Troyer (and (eight!) pals): [^].

Essentially, what Troyer et al. now say is that the D-Wave device does something that the classical devices apparently don’t, and so, the D-Wave device must be quantum! … If not all the classical devices, then at least the two devices: one, considered by they themselves, and the other, considered by Smolin and Smith. The D-Wave device behaves unlike both.

Further, Troyer et al. offer the following conjecture to account for the difference between the D-Wave chip and the [semi-]classical models:

“…The question of why SQA and semi-classical spin models correlate so differently with the D-Wave device is obviously important and interesting. We note that while SQA captures decoherence in the instantaneous energy eigenbasis of the system, so that each energy eigenstate—in particular the ground state—is itself a coherent superposition of computational basis states, semi-classical spin models assume that each qubit decoheres locally, thus removing all coherence from the ground state. We conjecture that the fact that the D-Wave machine succeeds with high probability on certain instances which the semi-classical models finds hard, can be understood in terms of this difference.”

[emphasis mine]

So, looks like, it is a quantum computer, after all. … At least, for this week!

* * *

Clearly, more studies required. So, here are a few questions to the QC research community:

What needs to be done to study the above conjecture more closely? Would some simple and special-purpose simulations that directly allow for a parametric control of the degrees of decoherence, help at least to illustrate (if not to fully support) the above conjecture? Such simulations could be highly simplified (say involving just a linear graph) but, still, sufficiently complete so as to be able to isolate, study, and possibly help settle, this issue.

How do you square off the quantum-ness of the D-Wave chip, and the “absence” of a speed-up, as discussed on Aaronson’s blog?

What measures would you suggest to capture the “percentage quantum-ness” of a QC? of an adiabatic quantum device such as D-Wave’s?

On these measures, how quantum are the current two D-Wave chips (D-Wave One and Two)? What is your estimate?

* * *

May be, more, later. (Who knows, it might once again collapse back to being a simple classical computer, next Monday!)

 

[May be I will come back (right today) and edit this post a bit, so as to make the write-up a bit more streamlined.]

 

[E&OE]