Machine “Learning”—An Entertainment [Industry] Edition

Yes, “Machine ‘Learning’,” too, has been one of my “research” interests for some time by now. … Machine learning, esp. ANN (Artificial Neural Networks), esp. Deep Learning. …

Yesterday, I wrote a comment about it at iMechanica. Though it was made in a certain technical context, today I thought that the comment could, perhaps, make sense to many of my general readers, too, if I supply a bit of context to it. So, let me report it here (after a bit of editing). But before coming to my comment, let me first give you the context in which it was made:


Context for my iMechanica comment:

It all began with a fellow iMechanician, one Mingchuan Wang, writing a post of the title “Is machine learning a research priority now in mechanics?” at iMechanica [^]. Biswajit Banerjee responded by pointing out that

“Machine learning includes a large set of techniques that can be summarized as curve fitting in high dimensional spaces. [snip] The usefulness of the new techniques [in machine learning] should not be underestimated.” [Emphasis mine.]

Then Biswajit had pointed out an arXiv paper [^] in which machine learning was reported as having produced some good DFT-like simulations for quantum mechanical simulations, too.

A word about DFT for those who (still) don’t know about it:

DFT, i.e. Density Functional Theory, is “formally exact description of a many-body quantum system through the density alone. In practice, approximations are necessary” [^]. DFT thus is a computational technique; it is used for simulating the electronic structure in quantum mechanical systems involving several hundreds of electrons (i.e. hundreds of atoms). Here is the obligatory link to the Wiki [^], though a better introduction perhaps appears here [(.PDF) ^]. Here is a StackExchange on its limitations [^].

Trivia: Kohn and Sham received a Physics Nobel for inventing DFT. It was a very, very rare instance of a Physics Nobel being awarded for an invention—not a discovery. But the Nobel committee, once again, turned out to have put old Nobel’s money in the right place. Even if the work itself was only an invention, it did directly led to a lot of discoveries in condensed matter physics! That was because DFT was fast—it was fast enough that it could bring the physics of the larger quantum systems within the scope of (any) study at all!

And now, it seems, Machine Learning has advanced enough to be able to produce results that are similar to DFT, but without using any QM theory at all! The computer does have to “learn” its “art” (i.e. “skill”), but it does so from the results of previous DFT-based simulations, not from the theory at the base of DFT. But once the computer does that—“learning”—and the paper shows that it is possible for computer to do that—it is able to compute very similar-looking simulations much, much faster than even the rather fast technique of DFT itself.

OK. Context over. Now here in the next section is my yesterday’s comment at iMechanica. (Also note that the previous exchange on this thread at iMechanica had occurred almost a year ago.) Since it has been edited quite a bit, I will not format it using a quotation block.


[An edited version of my comment begins]

A very late comment, but still, just because something struck me only this late… May as well share it….

I think that, as Biswajit points out, it’s a question of matching a technique to an application area where it is likely to be of “good enough” a fit.

I mean to say, consider fluid dynamics, and contrast it to QM.

In (C)FD, the nonlinearity present in the advective term is a major headache. As far as I can gather, this nonlinearity has all but been “proved” as the basic cause behind the phenomenon of turbulence. If so, using machine learning in CFD would be, by the simple-minded “analysis”, a basically hopeless endeavour. The very idea of using a potential presupposes differential linearity. Therefore, machine learning may be thought as viable in computational Quantum Mechanics (viz. DFT), but not in the more mundane, classical mechanical, CFD.

But then, consider the role of the BCs and the ICs in any simulation. It is true that if you don’t handle nonlinearities right, then as the simulation time progresses, errors are soon enough going to multiply (sort of), and lead to a blowup—or at least a dramatic departure from a realistic simulation.

But then, also notice that there still is some small but nonzero interval of time which has to pass before a really bad amplification of the errors actually begins to occur. Now what if a new “BC-IC” gets imposed right within that time-interval—the one which does show “good enough” an accuracy? In this case, you can expect the simulation to remain “sufficiently” realistic-looking for a long, very long time!

Something like that seems to have been the line of thought implicit in the results reported by this paper: [(.PDF) ^].

Machine learning seems to work even in CFD, because in an interactive session, a new “modified BC-IC” is every now and then is manually being introduced by none other than the end-user himself! And, the location of the modification is precisely the region from where the flow in the rest of the domain would get most dominantly affected during the subsequent, small, time evolution.

It’s somewhat like an electron rushing through a cloud chamber. By the uncertainty principle, the electron “path” sure begins to get hazy immediately after it is “measured” (i.e. absorbed and re-emitted) by a vapor molecule at a definite point in space. The uncertainty in the position grows quite rapidly. However, what actually happens in a cloud chamber is that, before this cone of haziness becomes too big, comes along another vapor molecule, and “zaps” i.e. “measures” the electron back on to a classical position. … After a rapid succession of such going-hazy-getting-zapped process, the end result turns out to be a very, very classical-looking (line-like) path—as if the electron always were only a particle, never a wave.

Conclusion? Be realistic about how smart the “dumb” “curve-fitting” involved in machine learning can at all get. Yet, at the same time, also remain open to all the application areas where it can be made it work—even including those areas where, “intuitively”, you wouldn’t expect it to have any chance to work!

[An edited version of my comment is over. Original here at iMechanica [^]]


 

“Boy, we seem to have covered a lot of STEM territory here… Mechanics, DFT, QM, CFD, nonlinearity. … But where is either the entertainment or the industry you had promised us in the title?”

You might be saying that….

Well, the CFD paper I cited above was about the entertainment industry. It was, in particular, about the computer games industry. Go check out SoHyeon Jeong’s Web site for more cool videos and graphics [^], all using machine learning.


And, here is another instance connected with entertainment, even though now I am going to make it (mostly) explanation-free.

Check out the following piece of art—a watercolor landscape of a monsoon-time but placid sea-side, in fact. Let me just say that a certain famous artist produced it; in any case, the style is plain unmistakable. … Can you name the artist simply by looking at it? See the picture below:

A sea beach in the monsoons. Watercolor.

If you are unable to name the artist, then check out this story here [^], and a previous story here [^].


A Song I Like:

And finally, to those who have always loved Beatles’ songs…

Here is one song which, I am sure, most of you had never heard before. In any case, it came to be distributed only recently. When and where was it recorded? For both the song and its recording details, check out this site: [^]. Here is another story about it: [^]. And, if you liked what you read (and heard), here is some more stuff of the same kind [^].


Endgame:

I am of the Opinion that 99% of the “modern” “artists” and “music composers” ought to be replaced by computers/robots/machines. Whaddya think?

[Credits: “Endgame” used to be the way Mukul Sharma would end his weekly Mindsport column in the yesteryears’ Sunday Times of India. (The column perhaps also used to appear in The Illustrated Weekly of India before ToI began running it; at least I have a vague recollection of something of that sort, though can’t be quite sure. … I would be a school-boy back then, when the Weekly perhaps ran it.)]

 

Advertisements

Paperity

If you are in one of the S&T fields and don’t know what “paperity” means, then guess it’s time you checked out the Web site: [^].

Came to know of it only today. Was doing some Web search on QM, and landed here [^]. Then, out of curiosity, also checked out an outgoing link [^] from that page, and thus, got the idea behind the site. … Hmmm… Need to explore it a bit more, but no time right now, so, may be, some time later!

Bye for now.


A Song I Like:

(Hindi) “saawan barse, tarse dil…”
Music: Aadesh Shrivastava, Anand Milind
Singers: Hariharan, Sadhana Sargam
Lyrics: Majrooh Sultanpuri

[TBD. May be tomorrow. Done right tonight (21:40 IST, 11 July 2017). Also corrected the spelling of “paperity” in the title and in the text.]

Miscellaneous: books to read, a new QM journal, the imposter syndrome, the US presidential elections

While my mood of not wanting to do anything in particular still continues (and also, there is no word yet on the job-related matters, including on whether I might qualify as a Professor of Mechanical Engineering in SPPU or not), there are a few quick things that I may as well note.

Updates on 17th, 18th and 22nd Nov. 2016: See my English translation[s] of the song, at the end of the post.


Books to Read:

First, the books to read. Here are a few books on my to-read list:

  1. Sean Carroll, “The Big Picture” [^]. I have been browsing through Sean’s blog-posts since before the time the book was published, and so have grown curious. I don’t have the money to buy it, right now, but once I get the next job, I sure plan to buy it. Here is the review in NY Times [^]. And, here is a latest review, written by a software engineer (whose link appeared in Sean’s twitter feed (I don’t myself use my Twitter account, but sometimes do check out the feeds of others via browser))[^]. Judging from his posts, I do know that Sean writes really well, and I would certainly want to check out this book, eventually.
  2. Roger Penrose, “Fashion, Faith, and Fantasy in the New Physics of the Universe,” [^]. This is the latest offering by Penrose. Sometimes I simply type “quantum physics” in Google, and then, in the search results, I switch the tab over to “news.” I came to know of this book via this route, last week, when I ran into this review [^].
  3. Roger Schlafly, “How Einstein Ruined Physics: Motion, Symmetry and Revolution in Science,” [^]. Here is a review [^], though my curiosity about the book rests not on the review but on two things: (i) what I had thought of Einstein myself, as far back as in early 1990s, while at UAB (hint: Schlafly’s thesis wouldn’t be out of bounds for me), and (ii) my reading the available portions of the book at Google Books. …This book has been on my “to-read” list for quite some time, but somehow it keeps slipping off. … Anyway, to be read, soon after I land a job…

A New QM Journal:

A new journal has arrived on the QM scene: [^]. Once again, I got to know of it through the “news” tab in a Google search on “quantum physics”, when I took this link [^].

It’s an arXiv-overlay journal. What it means is that first you submit your paper to arXiv. … As you know, getting something published at arXiv carries a pretty low bar (though it is not zero, and there have been some inconsistencies rarely reported about improper rejections even at arXiv). It’s good to bring your work to the notice of your peers, but it carries no value in your academic/research publications record, because arXiv is not a proper journal as such. … Now, if your work is good, you want to keep it open-access, but you don’t want to pay for keeping it open-access, and, at the same time, you also want to have the credentials of a proper journal publication to your credit, you have a solution, in the form of this arXiv-overlay journal. You send the link to your arXiv-published paper to them. If their editorial board finds it fitting the standards and purpose of their journal, they will include it.

The concept originated, I guess, with Timothy Gowers [^] and others’ efforts, when they started a maths journal called “Discrete Analysis.” At least I do remember reading about it last year [^]. Here is Gowers’ recent blog post reflecting on the success of this arXiv-overlay journal [^]. Here is what Nature had to report about the movement a few months ago [^].

How I wish there were an arXiv for engineering sciences too.

Especially in India, there has been a proliferation of bad journals: very poor quality, but they carry an ISSN, and they are accepted as journals in the Indian academia. I don’t have to take names; just check out the record of most any engineering professor from outside the IISc/IIT system, and you will immediately come to know what I mean.

At the same time, for graduate students, especially for the good PhD students who happen to lie outside the IIT system (there are quite a few such people), and for that matter even for MTech students in IITs, finding a good publication venue sometimes is difficult. Journal publications take time—1 or 2 years is common. Despite its size, population, or GDP, India hardly has any good journals being published from here. At the same time, India has a very large, sophisticated, IT industry.

Could this idea—arXiv-overlay journal—be carried into engineering space and in India? Could the Indian IT industry help in some ways—not just technical assistance in creating and maintaining the infrastructure, but also by way of financial assistance to do that?

We know the answer already in advance. But what the hell! What is the harm in at least mentioning it on a blog?


Just an Aside (re. QM): I spent some time noting down, on my mental scratch-pad, how QM should be presented, and in doing so, ended up with some rough outlines of  a new way to do so. I will write about it once I regain enough levels of enthusiasm.


The Imposter Syndrome:

It seems to have become fashionable to talk of the imposter syndrome [^]. The first time I read the term was while going through Prof. Abinandanan’s “nanopolitan” blog [^]. Turns out that it’s a pretty widely discussed topic [^], with one write-up even offering the great insight that “true imposters don’t suffer imposter syndrome” [^]. … I had smelled, albeit mildly, something like a leftist variety of a dead rat here… Anyway, at least writing about the phenomenon does seem to be prevalent among science-writers; here is a latest (H/T Sean Carroll’s feed) [^]…

Anyway, for the record: No, I have not ever suffered from the imposter syndrome, not even once in my life, nor do I expect to do so in future.

I don’t think the matter is big enough for me to spend any significant time analyzing it, but if you must (or if you somehow do end up analyzing it, for whatever reasons), here is a hint: In your work, include the concept of “standards,” and ask yourself just one question: does the author rest his standards in reason and reality, or does he do so in some people—which, in case of the imposter syndrome, would be: the other people.

Exercise: What (all) would stand opposite in meaning to the imposter syndrome? Do you agree with the suggestion here [^]?


The US Presidential Elections: Why are they so “big”? should they be?

Recently, I made a comment at Prof. Scott Aaronson’s blog, and at that time, I had thought that I would move it here as a separate post in its own right. However, I don’t think I have the energy right now, and once it returns, I am not sure if it will not get lost in the big stack of things to do. Anyway, here is the link [^]. … As I said, I am not interested much—if at all—in the US politics, but the question I dealt with was definitely a general one.


Overall, though, my mood of boredom continues… Yaawwwnnnn….


A Song I Like:

(Hindi) “seene mein jalan…”
Lyrics: Shahryar
Music: Jaidev
Singer: Suresh Wadkar

[Pune today is comparable to the Bombay of 1979 1978—but manages to stay less magnificent.]


Update on 2016.11.17: English translation of the song:

For my English blog-readers: A pretty good translation of the lyrics is available at Atul’s site; it is done by one Sudhir; see here [^]. This translation is much better than the English sub-titles appearing in this YouTube video [^] which comes as the first result when you google for this song. …

I am not completely happy with Sudhir’s translation (on Atul’s site) either, though it is pretty good. At a couple of places or so, it gives a slightly different shade of meaning than what the original Urdu words convey.

For instance, in the first stanza, instead of

“Just for that there is a heart inside,
one searches a pretext to be alive,”

it should be something like:

“just because there is a heart,
someone searches (i.e., people search) for an excuse which can justify its beating”

Similarly, in the second stanza,  instead of:

“what is this new intensity of loneliness, my friend?”,

a more accurate translation would be:

“what kind of a station in the journey of loneliness is this, my friends?”.

The Urdu word “manzil” means: parts of the Koran, and then, it has also come to mean: a stage in a journey, a station, a destination, or even a floor in a multi-storied building. But in no case does it mean intensity, as such. The underlying thought here is something like this: “loneliness is OK, but look, what kind of a lonely place it is that I have ended up in, my friends!” And the word for “friend” appears in the plural, not the singular. The song is one of a silent/quiet reflection; it is addressed to everyone in general and none in particular.

… Just a few things like that, but yes, speaking overall, Sudhir’s translation certainly is pretty good. Much better than what I could have done purely on my own, and in any case, it is strongly recommended. … The lyrics are an indispensable part of the soul of this song—in fact, the song is so damn well-integrated, all its elements are! So, do make sure to see Sudhir’s translation, too.


Update on 2016.11.18: My own English translation:

I have managed to complete my English translation of the above song. Let me share it with you. I benefitted a great deal from Sudhir’s translation and notes about the meanings of the words, mentioned in the note above, as well as further from “ek fankaar” [^]. My translation tries to closely follow not only the original words but also their sequence. To maintain continuity, the translation is given for the entire song as a piece.

First, the original Hindi/Urdu words:

seene mein jalan aankhon mein toofaan sa kyun hai
is shehar mein har shakhs pareshaan saa kyun hai

dil hai to dhadakne ka bahaanaa koi dhoondhe
patthar ki tarah behis-o-bejaan sa kyun hai

tanahaai ki ye kaun si manzil hai rafeeqon
ta-hadd-e-nazar ek bayaabaan saa kyon hai

kyaa koi nai baat nazar aati hai ham mein
aainaa hamen dekh ke hairaan sa kyon hai

Now, my English translation, with some punctuation added by me [and with further additions in the square brackets indicating either alternative words or my own interpolations]:

Why is there jealousy in the bosom; a tempest, as it were, in the eyes?
In this city, every person—why does it seem as if he were deeply troubled [or harassed]?

[It’s as if] Someone has a heart, so he might go on looking for an alibi [or a pretext] to justify [keeping it] beating
[But] A stone, as if it were that, why is it so numb and lifeless [in the first place]?

What kind of a station in the journey of the solitude is this, [my noble] friends?
Right to the end of the sight, why is there [nothing but] a sort of a total desolation?

Is there something new that has become visible about me?
The mirror, looking at me, why does it seem so bewildered [or perplexed]?

Update on 22nd Nov. 2016: OK, just one two more iterations I must have; just a slight change in the second [and the first [, and the third]] couplet[s]. (Even if further improvements would may be possible, I am now going to stop my iterations right here.):

Why is there jealousy in the bosom; a tempest, as it were, in the eyes?
In this city, every silhouette [of a person]—why does it seem as if he were deeply troubled [or harassed]?

[It’s as if] A heart, one does have, and so, someone might go on looking for an alibi [or a pretext] to justify [keeping it] beating
[But] A stone, as if it were that, why is it so numb and lifeless [in the first place]?

What kind of a station in the journey of the solitude is this, [my noble] friends?
[That] Right to the end of the sight, why is there [nothing but] a sort of a total desolation?

Is there something new that has become visible about me?
The mirror, looking at me, why does it seem so bewildered [or perplexed]?

 


[E&OE]

 

Food for Thought: A Few Recent arXiv Papers

arXiv

Before I wrote and published my preliminary QM papers in 2005 [^], I had done an extensive literature search, just to make sure that the sort of things that I was proposing had not already been published. arXiv.org came in especially handy because in physical sciences, it’s almost become a norm by now to dump one preprint over there even as the journal publication process continues.

In the past, I have written about how I used to be “followed up” in media, and also otherwise.  (…It still continues, but to a far lesser extent. It began lessening after the change of power at the Center (in India) in 2004, and esp. noticeably so, once the India-USA civilian nuclear deal went through.) But back in 2005 times and thereabouts, the “follow up” was still very much in force.

… Also, I had noticed a funny thing. The titles and contents of the submissions at arXiv would often seem to “talk” to each other. … I mean, if someone put a paper touching on an aether-based model, someone else just had to come and put his paper denying aether, and so on and so forth…

So, from around 2004 (roughly, my PhD registration) to about Oct 2009, I had made a habit of going over all the paper submissions in the “quant-ph” and “physics” categories at the arXiv.org.

I would go over each title, and if anything sounded related to my work by any stretch of imagination, I would make sure to go through the abstract and then, if necessary, also through the full paper. … As a rought estimate, I would go through the abstracts for about 25-30% of all the papers in the above two categories. Typically, I would end up downloading at least 25 papers each week, though most would be discarded. … I would do all of this just to be reasonably sure that my claims of novelty were meaningful, and of course, also to glean what other people were thinking on the same or related issues.

With my Ph.D. defense, I could sigh in relief… I mean I didn’t have that anxiety any more…

All in all, during this time—2004 through most of 2009—I spotted quite a few papers that were really interesting (to me). I would certainly like to share what these papers are, at some point in time. But it just so happens that all those downloads are now scattered in various directories and sub-directories on three separate HDDs of my home machines, and so, it’s going to be a big task by itself to collect all those interesting papers together… I definitely plan to do so one of these days but other things keep taking precedence.

Anyway, so… even if the defense is now over, I still have this habit of visiting arXiv every now and then (though I don’t go over each and every submission these days).

A Couple of Recent Articles

Last week or so, I found two interesting articles that provide good food for thought, and would like to share these with you.

One of these is: Dragoljub A. Cucic,  “Types of paradox in physics,” arXiv:0912.1864v1 [^]. It’s a very comprehensive kind of article. Impressed, I did an author search on Cucic, and found a few more papers on this topic by him [^].

The other article I have in mind is: Franco Bagnoli, “From Newton to cellular automata,” arXiv:0912.2056v1 [^]. Again, the scope of this article is just wonderfully wide, even though the writing tends to be a bit too terse at places. But Bagnoli compensates for this by including a neat “concept map.”

Both the papers are easily accessible even to undergraduates. Both provide enormous food for thought.

Indeed, I already find myself wondering if I should write an article or two addressing one or two of the many paradoxes that Cucic lists.

And, I cannot thank Bagnoli enough for providing a kind of “white paper” material that was so badly needed in explaining to other researchers (not just to laymen) just what kind of research ideas and methods I seem to be pursuing and how these differ from those in the typical PhD researches, esp. those from the engineering sciences. It helps explain why there is this  general (and pretty vague) impression to the effect that there is not enough “maths” or “rigour” in my research or in my papers… Bagnoli helps point out the why of it…

I might even write an informal article showing what kind of maths it will look like if an artificial attempt is made to mathematicize these ideas at any cost, using only the classical or traditional way of putting maths… [I would write such an article anyway but especially so if some renowned scientist/mathematician has problems accepting my research otherwise and so asks me to do so [as I had indicated in my last post (just below).]

Anyway, do go over these articles, and if you wish to share a comment or two, feel free to write.  [I also plan to start a thread at iMechanica for discussing these papers.]

– – – – –

Two Songs I Like [more or less at random]

1. (Hindi) “hawaon pe likh do, hawaon ke naam…”
Singer: Kishore Kumar
Lyrics: Gulzar
Music: Hemant Kumar

2. (Marathi) “maanasichaa chitrakaar to…”
Singer: Hridaynath Mangeshkar
Music: Vasant Prabhu (?)
Lyrics: P. Savalaram (?)