# Some running thoughts on ANNs and AI—1

Go, see if you want to have fun with the attached write-up on ANNs [^] (but please also note the version time carefully—the write-up could change without any separate announcement).

The write-up is more in the nature of a very informal blabber of the kind that goes when people work out something on a research blackboard (or while mentioning something about their research to friends, or during brain-storming session, or while jotting things on the back of the envelop, or something similar).

A “song” I don’t like:

(Marathi) “aawaaj waaDaw DJ…”
“Credits”: Go, figure [^]. E.g., here [^]. Yes, the video too is (very strongly) recommended.

Update on 05 October 2018 10:31 IST:

Psychic attack on 05 October 2018 at around 00:40 IST (i.e. the night between 4th and 5th October, IST).

/

# Off the blog. [“Matter” cannot act “where” it is not.]

I am going to go off the blogging activity in general, and this blog in most particular, for some time. [And, this time round, I will keep my promise.]

The reason is, I’ve just received the shipment of a book which I had ordered about a month ago. Though only about 300 pages in length, it’s going to take me weeks to complete. And, the book is gripping enough, and the issue important enough, that I am not going to let a mere blog or two—or the entire Internet—come in the way.

I had read it once, almost cover-to-cover, some 25 years ago, while I was a student in UAB.

Reading a book cover-to-cover—I mean: in-sequence, and by that I mean: starting from the front-cover and going through the pages in the same sequence as the one in which the book has been written, all the way to the back-cover—was quite odd a thing to have happened with me, at that time. It was quite unlike my usual habits whereby I am more or less always randomly jumping around in a book, even while reading one for the very first time.

But this book was different; it was extraordinarily engaging.

In fact, as I vividly remember, I had just idly picked up this book off a shelf from the Hill library of UAB, for a casual examination, had browsed it a bit, and then had began sampling some passage from nowhere in the middle of the book while standing in an library aisle. Then, some little time later, I was engrossed in reading it—with a folded elbow resting on the shelf, head turned down and resting against a shelf rack (due to a general weakness due to a physical hunger which I was ignoring [and I would have have to go home and cook something for myself; there was none to do that for me; and so, it was easy enough to ignore the hunger]). I don’t honestly remember how the pages turned. But I do remember that I must have already finished some 15-20 pages (all “in-the-order”!) before I even realized that I had been reading this book while still awkwardly resting against that shelf-rack. …

… I checked out the book, and once home [student dormitory], began reading it starting from the very first page. … I took time, days, perhaps weeks. But whatever the length of time that I did take, with this book, I didn’t have to jump around the pages.

The issue that the book dealt with was:

[Instantaneous] Action at a Distance.

The book in question was:

Hesse, Mary B. (1961) “Forces and Fields: The concept of Action at a Distance in the history of physics,” Philosophical Library, Edinburgh and New York.

It was the very first book I had found, I even today distinctly remember, in which someone—someone, anyone, other than me—had cared to think about the issues like the IAD, the concepts like fields and point particles—and had tried to trace their physical roots, to understand the physical origins behind these (and such) mathematical concepts. (And, had chosen to say “concepts” while meaning ones, rather than trying to hide behind poor substitute words like “ideas”, “experiences”, “issues”, “models”, etc.)

But now coming to Hesse’s writing style, let me quote a passage from one of her research papers. I ran into this paper only recently, last month (in July 2017), and it was while going through it that I happened [once again] to remember her book. Since I did have some money in hand, I did immediately decide to order my copy of this book.

Anyway, the paper I have in mind is this:

Hesse, Mary B. (1955) “Action at a Distance in Classical Physics,” Isis, Vol. 46, No. 4 (Dec., 1955), pp. 337–353, University of Chicago Press/The History of Science Society.

The paper (it has no abstract) begins thus:

The scholastic axiom that “matter cannot act where it is not” is one of the very general metaphysical principles found in science before the seventeenth century which retain their relevance for scientific theory even when the metaphysics itself has been discarded. Other such principles have been fruitful in the development of physics: for example, the “conservation of motion” stated by Descartes and Leibniz, which was generalized and given precision in the nineteenth century as the doctrine of the conservation of energy; …

Here is another passage, once again, from the same paper:

Now Faraday uses a terminology in speaking about the lines of force which is derived from the idea of a bundle of elastic strings stretched under tension from point to point of the field. Thus he speaks of “tension” and “the number of lines” cut by a body moving in the field. Remembering his discussion about contiguous particles of a dielectric medium, one must think of the strings as stretching from one particle of the medium to the next in a straight line, the distance between particles being so small that the line appears as a smooth curve. How seriously does he take this model? Certainly the bundle of elastic strings is nothing like those one can buy at the store. The “number of lines” does not refer to a definite number of discrete material entities, but to the amount of force exerted over a given area in the field. It would not make sense to assign points through which a line passes and points which are free from a line. The field of force is continuous.

See the flow of the writing? the authentic respect for the intellectual history, and yet, the overriding concern for having to reach a conclusion, a meaning? the appreciation for the subtle drama? the clarity of thought, of expression?

Well, these passages were from the paper, but the book itself, too, is similarly written.

Obviously, while I remain engaged in [re-]reading the book [after a gap of 25 years], don’t expect me to blog.

After all, even I cannot act “where” I am not.

A Song I Like:

[I thought a bit between this song and another song, one by R.D. Burman, Gulzar and Lata. In the end, it was this song which won out. As usual, in making my decision, the reference was exclusively made to the respective audio tracks. In fact, in the making of this decision, I happened to have also ignored even the excellent guitar pieces in this song, and the orchestration in general in both. The words and the tune were too well “fused” together in this song; that’s why. I do promise you to run the RD song once I return. In the meanwhile, I don’t at all mind keeping you guessing. Happy guessing!]

(Hindi) “bheegi bheegi…” [“bheege bheege lamhon kee bheegee bheegee yaadein…”]
Music and Lyrics: Kaushal S. Inamdar
Singer: Hamsika Iyer

[Minor additions/editing may follow tomorrow or so.]

/

# On whether A is not non-A

This post has its origin in a neat comment I received on my last post [^]; see the exchange starting here: [^].

The question is whether I accept that A is not non-A.

My answer is: No, I do not accept that, logically speaking, A is not non-A—not unless the context to accept this statement is understood clearly and unambiguously (and the best way to do that is to spell it out explicitly).

Another way to say the same thing is that I can accept that “A is not non-A,” but only after applying proper qualifications; I won’t accept it in an unqualified way.

Let me explain by considering various cases arising, using a simple example.

The Venn diagram:

Let’s begin by drawing a Venn diagram.

Draw a rectangle and call it the set $R$. Draw a circle completely contained in it, and call it the set $A$. You can’t put a round peg to fill a rectangular hole, so, the remaining area of the rectangle is not zero. Call the remaining area $B$. See the diagram below.

Case 1: All sets are non-empty:

Assume that neither $A$ nor $B$ is empty. Using symbolic terms, we can say that:
$A \neq \emptyset$,
$B \neq \emptyset$, and
$R \equiv A \cup B$
where the symbol $\emptyset$ denotes an empty set, and $\equiv$ means “is defined as.”

We take $R$ as the universal set—of this context. For example, $R$ may represent, say the set of all the computers you own, with $A$ denoting your laptops and $B$ denoting your desktops.

I take the term “proper set” to mean a set that has at least one element or member in it, i.e., a set which is not empty.

Now, focus on $A$. Since the set $A$ is a proper set, then it is meaningful to apply the negation- or complement-operator to it. [May be, I have given away my complete answer right here…] Denote the resulting set, the non-A, as $A^{\complement }$. Then, in symbolic terms:
$A^{\complement } \equiv R \setminus A$.
where the symbol $\setminus$ denotes taking the complement of the second operand, in the context of the first operand (i.e., “subtracting” $A$ from $R$). In our example,
$A^{\complement } = B$,
and so:
$A^{\complement } \neq \emptyset$.
Thus, here, $A^{\complement }$ also is a proper (i.e. non-empty) set.

To conclude this part, the words “non-A”, when translated into symbolic terms, means $A^{\complement }$, and this set here is exactly the same as $B$.

To find the meaning of the phrase “not non-A,” I presume that it means applying the negation i.e. the complement operator to the set $A^{\complement }$.

It is possible to apply the complement operator because $A ^{\complement } \neq \emptyset$. Let us define the result of this operation as $A^{\complement \complement}$; note the two $^{\complement}$s appearing in its name. The operation, in symbols becomes:
$A^{\complement \complement} \equiv R \setminus A^{\complement} = R \setminus B = A$.
Note that we could apply the complement operator to $A$ and later on to $A^{\complement}$ only because each was non-empty.

As the simple algebra of the above simple-minded example shows,
$A = A^{\complement\complement}$,
which means, we have to accept, in this example, that A is not non-A.

Remarks on the Case 1:

However, note that we can accept the proposition only under the given assumptions.

In  particular, in arriving at it, we have applied the complement-operator twice. (i) First, we applied it to the “innermost” operand i.e. $A$, which gave us $A^{\complement}$. (ii) Then, we took this result, and applied the complement-operator to it once again, yielding $A^{\complement\complement}$. Thus, the operand for the second complement-operator was $A^{\complement}$.

Now, here is the rule:

Rule 1: We cannot meaningfully apply the complement-operator unless the operand set is proper (i.e. non-empty).

People probably make mistakes in deciding whether A is not non-A, because, probably, they informally (and properly) do take the “innermost” operand, viz. $A$, to be non-empty. But then, further down the line, they do not check whether the second operand, viz. $A^{\complement}$ turns out to be empty or not.

Case 2: When the set $A^{\complement}$ is empty:

The set $A^{\complement}$ will be empty if $B = \emptyset$, which will happen if and only if $A = R$. Recall, $R$ is defined to be the union of $A$ and $B$.

So, every time there are two mutually exclusive and collectively exhaustive sets, if any one of them is made empty, you cannot doubly apply the negation or the complement operator to the other (nonempty) set.

Such a situation always occurs whenever the remaining set coincides with the universal set of a given context.

In attempting a double negation, if your first (or innermost) operand itself is a universal set, then you cannot apply the negation operator for the second time, because by Rule 1, the result of the first operator comes out as an empty set.

The nature of an empty set:

But why this rule that you can’t negate (or take the complement of) an empty set?

An empty set contains no element (or member). Since it is the elements which together impart identity to a set, an empty set has no identity of its own.

As an aside, some people think that all the usages of the phrase “empty set” refers to the one and the only set (in the entire universe, for all possible logical propositions involving sets). For instance, the empty set obtained by taking an intersection of dogs and cats, they say, is exactly the same empty set as the one obtained by taking an intersection of cars and bikes.

I reject this position. It seems to me to be Platonic in nature, and there is no reason to give Plato even an inch of the wedge-space in this Aristotlean universe of logic and reality.

As a clarification, notice, we are talking of the basic and universal logic here, not the implementation details of a programming language. A programming language may choose to point all the occurrences of the NULL string to the same memory location. This is merely an implementation choice to save on the limited computer memory. But it still makes no sense to say that all empty C-strings exist at the same memory location—but that’s what you end up having if you call an empty set the empty set. Which brings us to the next issue.

If an empty set has no identity of its own, if it has no elements, and hence no referents, then how come it can at all be defined? After all, a definition requires identity.

The answer is: Structurally speaking, an empty set acquires its meaning—its identity—“externally;” it has no “internally” generated identity.

The only identity applicable to an empty set is an abstract one which gets imparted to it externally; the purpose of this identity is to bring a logical closure (or logical completeness) to the primitive operations defined on sets.

For instance, intersection is an operator. To formally bring closure to the intersection operation, we have to acknowledge that it may operate over any combination of any operand sets, regardless of their natures. This range includes having to define the intersection operator for two sets that have no element in common. We abstractly define the result of such a case as an empty set. In this case, the meaning of the empty set refers not to a result set of a specific internal identity, but only to the operation and the disjoint nature the operands which together generated it, i.e., via a logical relation whose meaning is external to the contents of the empty set.

Inasmuch as an empty set necessarily includes a reference to an operation, it is a concept of method. Inasmuch as many combinations of various operations and operands can together give rise to numerous particular instances of an empty set, there cannot be a unique instance of it which is applicable in all contexts. In other words, an empty set is not a singleton; it is wrong to call it the empty set.

Since an empty set has no identity of its own, the notion cannot be applied in an existence-related (or ontic or metaphysical) sense. The only sense it has is in the methodological (or epistemic) sense.

Extending the meaning of operations on an empty set:

In a derivative sense, we may redefine (i.e. extend) our terms.

First, we observe that since an empty set lacks an identity of its own, the result of any operator applied to it cannot have any (internal) identity of its own. Then, equating these two lacks of existence-related identities (which is where the extension of the meaning occurs), we may say, even if only in a derivative or secondary sense, that

Rule 2: The result of an operator applied to an empty set again is another empty set.

Thus, if we now allow the complement-operator to operate also on an empty set (which, earlier, we did not allow), then the result would have to be another empty set.

Again, the meaning of this second empty set depends on the entirety of its generating context.

Case 3: When the non-empty set is the universal set:

For our particular example, assuming $B = \emptyset$ and hence $A = R$, if we allow complement operator to be applied (in the extended sense) to $A^{\complement}$, then

$A^{\complement\complement} \equiv R \setminus A^{\complement} = R \setminus (R \setminus A) = R \setminus B = R \setminus (\emptyset) = R = A$.

Carefully note, in the above sequence, the place where the extended theory kicks in is at the expression: $R \setminus (\emptyset)$.

We can apply the $\setminus$ operator here only in an extended sense, not primary.

We could here perform this operation only because the left hand-side operand for the complement operator, viz., the set $R$ here was a universal set. Any time you have a universal set on the left hand-side of a complement operator, there is no more any scope left for ambiguity. This state is irrespective of whether the operand on the right hand-side is a proper set or an empty set.

So, in this extended sense, feel free to say that A is not non-A, provided A is the universal set for a given context.

To recap:

The idea of an empty set acquires meaning only externally, i.e., only in reference to some other non-empty set(s). An empty set is thus only an abstract place-holder for the result of an operation applied to proper set(s), the operation being such that it yields no elements. It is a place-holder because it refers to the result of an operation; it is abstract, because this result has no element, hence no internally generated identity, hence no concrete meaning except in an abstract relation to that specific operation (including those specific operands). There is no “the” empty set; each empty set, despite being abstract, refers to a combination of an instance of proper set(s) and an instance of an operation giving rise to it.

Exercises:

E1: Draw a rectangle and put three non-overlapping circles completely contained in it. The circles respectively represent the three sets $A$, $B$, $C$, and the remaining portion of the rectangle represents the fourth set $D$. Assuming this Venn diagram, determine the meaning of the following expressions:

(i) $R \setminus (B \cup C)$ (ii) $R \setminus (B \cap C)$ (iii) $R \setminus (A \cup B \cup C)$ (iv) $R \setminus (A \cap B \cap C)$.

(v)–(viii) Repeat (i)–(iv) by substituting $D$ in place of $R$.

(ix)–(xvi) Repeat (i)–(viii) if $A$ and $B$ partly overlap.

E2: Identify the nature of set theoretical relations implied by that simple rule of algebra which states that two negatives make a positive.

A bit philosophical, and a form better than “A is not non-A”:

When Aristotle said that “A is A,” and when Ayn Rand taught its proper meaning: “Existence is identity,” they referred to the concepts of “existence” and “identity.” Thus, they referred to the universals. Here, the word “universals” is to be taken in the sense of a conceptual abstraction.

If concepts—any concepts, not necessarily only the philosophical axioms—are to be represented in terms of the set theory, how can we proceed doing that?

(BTW, I reject the position that the set theory, even the so-called axiomatic set theory, is more fundamental than the philosophic abstractions.)

Before we address this issue of representation, understand that there are two ways in which we can specify a set: (i) by enumeration, i.e. by listing out all its (relatively concrete) members, and (ii) by rule, i.e. by specifying a definition (which may denote an infinity of concretes of a certain kind, within a certain range of measurements).

The virtue of the set theory is that it can be applied equally well to both finite sets and infinite sets.

The finite sets can always be completely specified via enumeration, at least in principle. On the other hand, infinite sets can never be completely specified via enumeration. (An infinite set is one that has an infinity of members or elements.)

A concept (any concept, whether of maths, or art, or engineering, or philosophy…) by definition stands for an infinity of concretes. Now, in the set theory, an infinity of concretes can be specified only using a rule.

Therefore, the only set-theoretic means capable of representing concepts in that theory is to specify their meaning via “rule” i.e. definition of the concept.

Now, consider for a moment a philosophical axiom such as the concept of “existence.” Since the only possible set-theoretic representation of a concept is as an infinite set, and since philosophical axiomatic concepts have no antecedents, no priors, the set-theoretic representation of the axiom of “existence” would necessarily be as a universal set.

We saw that the complement of a universal set is an empty set. This is a set-theoretic conclusion. Its broader-based, philosophic analog is: there are no contraries to axiomatic concepts.

For the reasons explained above, you may thus conclude, in the derivative sense, that:

“existence is not void”,

where “void” is taken as exactly synonymous to “non-existence”.

The proposition quoted in the last sentence is true.

However, as the set theory makes it clear and easy to understand, it does not mean that you can take this formulation for a definition of the concept of existence. The term “void” here has no independent existence; it can be defined only by a negation of existence itself.

You cannot locate the meaning of existence in reference to void, even if it is true that “existence is not void”.

Even if you use the terms in an extended sense and thereby do apply the “not” qualfier (in the set-theoretic representation, it would be an operator) to the void (to the empty set), for the above-mentioned reasons, you still cannot then read the term “is” to mean “is defined as,” or “is completely synonymous with.” Not just our philosophical knowledge but even its narrower set-theoretical representation is powerful enough that it doesn’t allow us doing so.

That’s why a better way to connect “existence” with “void” is to instead say:

“Existence is not just the absence of the void.”

The same principle applies to any concept, not just to the most fundamental philosophic axioms, so long as you are careful to delineate and delimit the context—and as we saw, the most crucial element here is the universal set. You can take a complement of an empty set only when the left hand-side operator is a universal set.

Let us consider a few concepts, and compare putting them in the two forms:

• from “A is not non-A”
• to “A is not the [just] absence [or negation] of non-A,” or, “A is much more than just a negation of the non-A”.

Consider the concept: focus. Following the first form, a statement we can formulate is:

“focus is not evasion.”

However, it does make much more sense to say that

“focus is not just an absence of evasion,” or that “focus is not limited to an anti-evasion process.”

Both these statements follow the second form. The first form, even if it is logically true, is not as illuminating as is the second.

Exercises:

Here are a few sentences formulated in the first form—i.e. in the form “A is not non-A” or something similar. Reformulate them into the second form—i.e. in the form such as: “A is not just an absence or negation of non-A” or “A is much better than or much more than just a complement or negation of non-A”. (Note: SPPU means the Savitribai Phule Pune University):

• Engineers are not mathematicians
• C++ programmers are not kids
• IISc Bangalore is not SPPU
• IIT Madras is not SPPU
• IIT Kanpur is not SPPU
• IIT Bombay is not SPPU
• The University of Mumbai is not SPPU
• The Shivaji University is not SPPU

[Lest someone from SPPU choose for his examples the statements “Mechanical Engg. is not Metallurgy” and “Metallurgy is not Mechanical Engg.,” we would suggest him another exercise, one which would be better suited to the universal set of all his intellectual means. The exercise involves operations mostly on the finite sets alone. We would ask him to verify (and not to find out in the first place) whether the finite set (specified with an indicative enumeration) consisting of {CFD, Fluid Mechanics, Heat Transfer, Thermodynamics, Strength of Materials, FEM, Stress Analysis, NDT, Failure Analysis,…} represents an intersection of Mechanical Engg and Metallurgy or not.]

A Song I Like:

[I had run this song way back in 2011, but now want to run it again.]

(Hindi) “are nahin nahin nahin nahin, nahin nahin, koee tumasaa hanseen…”
Singers: Kishore Kumar, Asha Bhosale
Music: Rajesh Roshan
Lyrics: Anand Bakshi

[But I won’t disappoint you. Here is another song I like and one I haven’t run so far.]

(Hindi) “baaghon mein bahaar hain…”
Music: S. D. Burman [but it sounds so much like R.D., too!]
Singers: Mohamad Rafi, Lata Mangeshkar
Lyrics: Anand Bakshi

[Exercise, again!: For each song, whenever a no’s-containing line comes up, count the number of no’s in it. Then figure out whether the rule that double negatives cancel out applies or not. Why or why not?]

[Mostly done. Done editing now (right on 2016.10.22). Drop me a line if something isn’t clear—logic is a difficult topic to write on.]

[E&OE]