[Largely] Unjustified Comments on the IIT JEE

Here are some of the initial thoughts that passed my mind while reading the recent news items concerning the proposed scraping of the IIT JEE. I noticed this development when Dr. T. A. Abinandanan pointed the news via posts here [^] and here [^],  and provided a link to the original The Telegraph news item here [^]. Also see certain IITians’ petition against this decision that Abi referred to, here [^].

An explanation about the title of this post: I call my comments “unjustified” not because these cannot be justified but only because I do not explicitly provide adequate level of justification for them here. And then, I searched for a word to stand for what I had in mind, but didn’t find any. … Anyway: don’t expect an essay-like write-up here; expect only a few points towards jotting down my positions, and some of the related things that strike me. … BTW, I am not a professional writer—only an engineer (in the sense: engineer-cum-physicist), and, for that matter, both my composition skills and English are poor.

Thus having [skillfully] dodged all the responsibility and having absolved myself of all the consequences, here I go:

1. In a proper government, there is no right to education—there are only the individual rights.

For more information (actually, enlightenment) on the idea of individual rights, go to Ayn Rand. [If you don’t have the brains to know how to or where to find the writings by her, about her, or the writings by people qualified to be called her students/followers, stop reading this blog post.] Her writings is the broad context I assume here.

2. In a properly constituted and governed nation, there are no government-run entrance examinations—whether a single examination for a single institute, or a single examination for several institutes bunched together under a single “brand” owned by the government, or several examinations for single/several institutes.

The entire domain of education—like every sphere of production and trade—is left completely free of the coercive government controls. In the context of education, the freedom includes that for founding and running universities including sub-tasks such as hiring teachers/professors and administrators to run them, designing programs and setting syllabus, conducting examinations and evaluating performance, and giving out certificates/degrees. Also, admitting students, and the criteria applied while doing so. Not a single aspect or activity in education would have even a shadow of government controls.

I hear that parents are willing to shell out lakhs of Rupees on coaching classes. Reputations can be, and indeed are, built that way—without government help (actually, hindrance).

3. It’s possible to advocate a gradual lessening of the improper government controls. I mean this statement only in one particular sense: advocating a course of ever-decreasing government controls, one in which government comes to cease interfering with economy (including services) within a finite and reasonable time-frame, cannot be taken as compromising on basic principles of free markets (i.e. laissez-faire capitalism in Ayn Rand’s sense of the term).

4. It’s ridiculous for today’s educated Indian not to see through the decidedly socialistic nature of  “brand-“building for IITs (including the IIT JEE). It’s even more ridiculous if a JEE IITian fails to see it. (By JEE IITian, I mean those graduates of IITs who had entered the institutes via the JEE route, as against say, the GATE or any other route.) I say so on several counts: (i) IITs are owned by the government, (ii) JEE is conducted by the government, (iii) the concept of brand has meaning only in the context of free markets.

5. It is improper to attempt to gain support for the idea of a government-run nationwide entrance examination, by making appeal to Olympiads/SAT/GRE etc. The propensity to make comparisons with the specifically 20th (and worse: 21st) century American institutions is nothing but a symptom indicating the basic inability to think in terms of principles. This is one ability that should have been cultivated by any purportedly good education system, e.g. IITs, but evidently doesn’t seem to be.

6. It’s completely shameless on the part of the JEE IITian to try to elevate the JEE brand even above the standards indicated by “the best in/from India”.

The JEE IITian would do well to remember the last time he, his father, or better still, his father-in-law, had received a world-class medical opinion, may be, even a surgery, in India. Then, he should consider the fact that the MBBS program has never had a JEE; that all admissions have always been made on the basis of XII board (or similar) examinations.

Yes, he might also wish to consider statistical aspects—the distribution function for the best to the mediocre among the medical graduates produced by a system that always had board examinations as its input criterion. The results may still surprise him.

I won’t point out the profile of engineers produced by the “other” colleges to a typical JEE IITian, because I know through my personal experience that it would be impossible to overcome the inestimable heights of his JEE-bigotry barrier. But, yes, the case of MBBS might be acknowledged by him to be a bit different.

7. Still, it must be noted that as many as 35% to 50% of the JEE IITian fail to even merely qualify at GATE.

Take a moment to realize what this means. Suppose that 3000 JEE pass folks were admitted to the IITs about 5–10 years ago (the intake capacity was comparatively smaller back then). Almost every one of these 3,000 might proudly tell you that he had had an AIR of within 3,000 out of, say, about 3 lakh aspirants—a selection ratio of 1 in 100 and whatnot. (Refer to their sites—even the official IIT sites—for more hype of this sort.)

Now assume that, putting all branches together, about 1 lakh students sat for GATE. (The figures are like 20 to 30 thousand candidates each for bigger branches like Mechanical and Civil, and so, the total figure of 1 lakh should not be too off the mark.)

What these IITian chappies—and their Professors, Deans, and Directors—won’t tell you is this: about one in every two or three of this “cream of cream” fails to fall within the top 30,000 rank.

And this abysmal performance arises despite the following facts: (i) going by their own words, they had been given the very best of education, (ii) in any case they had far more resources spent on them (perhaps a factor—not percentage—of 10, 25, or more), and (iii) perhaps worst: these IITians, unlike the students coming from “other” engineering colleges had ample opportunity to get used to the style and difficulty level for the kind of questions that would be set at GATE. After all, GATE question papers are set (and evaluated) by the same (or the same kind of) professors who had taught these IITians for as many as 4 years.

8. No matter what be the content, format, or method of conducting an examination, scores produced by any examination will always be, at best, only estimates of the true ability profile.

I plan to post the extent of correlation existing between university BE marks and GATE scores some time in future. (In advance, let me say this much: GATE paper questions tend to be better than those at the typical university examinations. And, the correlation tends to be poor. So much for the “great grades all throughout.” Never confuse persistence or continuity of the effort on the one hand and the consistency of “straight A” grades on the other.)

But coming back to the “estimate” thing. The issue here is not that there is some sort of an inherent epistemological  “uncertainty” to scores or evaluations. Or that there is an in-principle inability to precisely determine the objective worth of a candidate.  Not that. The issue here is: what you mean by “precisely.”

There is always a finite band to the results of any examination. The band arises due to sampling—not statistics per say, but because of the fact that the actual examination is only a sub-set of all possible questions that can properly be asked. Therefore, the same candidate, with exactly the same preparation (and keeping aside any other variables such as mental mood, energy, etc.) will still perform differently, purely because the profile of the knowledge acquired by him is not uniform across all sub-topics. One question may emphasize one sub-sub-sub topic that the candidate has in fact understood somewhat better than another sub-sub-sub topic. These ups and downs in the level of understanding are always present. They remain there right up to the very top level of performance. (The existence of unevenness of grasp across topics is true even for those people who score a perfect 100/100 marks on examinations.) The band I had in mind actually refers to this kind of a range. To believe that it don’t exist is to fool yourself—which is what most students (esp. those who excel), parents, teachers, administrators, employers, and all are busy doing all the time.

Here I wish to point out that the GRE booklets used to acknowledge that there was a “band” to their scores. Firstly, they would point out that your scores could improve by several hundred points if you were to invest enough time in preparation and practice. Secondly, as a separate point (and this indeed is a separate point), they would also point out that the score could easily be different by a range of about +/- 30-40 score points over the range of the maximum 800 points. I don’t remember if they would gave the range in terms of std deviation or not. But I do remember that +/- 30-40 points was the extent of the range.  Notice how huge that range—that band—is: for a guy with a 750 score, this range implies that his score could have been as low as 710 and as high as 790.  A lot of change occurs over such a range. May be, it’s as big as going from 92 percentile up to 98 percentile or down to 86 percentile.

The ETS people were honest.

Would the JEE IITians go ahead and identify (in the standard deviations terms or in any other) the band or the range for their scores at the higher end?

9. Treating all board examinations to be completely at par with each other commits the offense of egalitarianism.

The injury does not go away even if you do some jumping around through the statistical hoops—say via “percentile score, normalized via doubly conducted higher-order multi-parameter regression analysis.” It all still might mean that a brilliant student from Maharashtra is displaced by a lesser guy from Uttar Pradesh.

Yes, BITS Pilani has always been doing this. But then, remember: the Birlas are not the Government of India. And, less importantly: BITS Pilani graduates are not known to rest their reputations on the performance they had displayed before getting into their alma matter. The JEE IITians, in contrast, regularly do.

10. The taste of the pudding is in the eating.

Education is not the same as a recreational game. Education is supposed to be a means to help you live your life better—by expanding/improving your efficacy—by expanding your conceptual abilities (or knowledge) in terms of both contents and skills.

. . . . .

May be, more on this topic, later.

– – – – –

A Song I Like:

(Hindi) “waadiyaan meraa daaman, raaste meri baahen..”
Music: R. D. Burman
Singer: Mohammad Rafi and Lata Mangeshkar
Lyrics: Majrooh Sultanpuri

[PS: Updated on April 19, 2010, by inserting what now is point # 7. Also to be further improved some time later… BTW, ]

Advertisements

8 thoughts on “[Largely] Unjustified Comments on the IIT JEE

  1. Excellent points and a wonderful post.

    I think that the JEE should be gotten rid of. In its place, I suggest that they have an examination in which anyone in the top 10 percent of his or her school is able to appear. The exam will be on a topic that is chosen by the student outside the math, phy, chem stream. For instance, the subjects could be anthropology, molecular biology, economics, astronomy, psychology, etc.

    You have to score 50 percent (or some such easy cut-off) in the exam. That gets you in the “Qualified Pool of Candidates.” Now comes the interesting bit. Randomly draw N names from this pool, where N is the capacity of the IITs. Randomly assign a rank to each name drawn and offer choice of the institution they wish to attend based on the rank.

    What will this do? It will force kids to do well in school. The multi-year coaching for JEE nonsense will be gone. The students will learn a subject outside their own discipline. The random draw is just the luck of the draw — which is not discriminatory. A kid who went to a not-so-fancy school has almost the same chance as any other kid.

    Discussions welcome.

    • Hi Atanu,

      The first time I read your comment, I started LOL! … But then, soon later, I realized that your suggestion does weave together most every important criticism regarding the IIT JEE and all…

      1. Ok. First things first. I think my biggest points here (whether stated explicitly above or not) are that (i) IITs should be privatized, and (ii) this idea of having a government-run super-examination—which actually accords so well with Plato’s blatantly statist ideas—should be summarily done away with in every form. There should not be any super-examination, and for that matter, even a super-criterion. Each institute should be left to decide its own admission criteria, in a free-market manner. If they fail in doing this part of their job, that will reflect on their reputation, and the cumulative free choices of the free consumers of the education market, would together go towards appropriately punishing it.

      2. With that said, still, a couple of questions can be raised: (i) In a free society, can’t there be a consortium/association of engineering colleges that might, together, find it effective/efficient to have a common entrance examination or set of criteria? (ii) What can be said about the IIT JEE’s standards in reference to such an hypothetical examination/criteria? … I think what you say goes towards these two questions. … Without bothering to be systematic about it, let me jot down a few points as they strike me.

      3. Considering the following statistics: 3 lakh JEE aspirants, 3000 combined admissions to IITs, and four years later, 1 lakh GATE aspirants and a one-third or one-half of the JEE IITians unable to fall within 30,000, it makes for a very strong case for your suggestion that the top 10 percent from school be allowed. The percentage would be perfectly just.

      4. On the first reading, I thought that the 4th subject was a wonderful idea. But then, soon enough, I recalled the rapidity with which the GRE coaching classes in the city were able to introduce coaching for analytical writing for the new GRE format. The idea of encouraging students to master an “out of the box” subject is wonderful, but we all know what the Indian ingenuity to beat the system would make of it.

      5. Random draws. This was the portion where I had started LOL! Hmmm… Going back to point 3. of this reply, again, it does seem like a good ballpark idea. But then, a detailed consideration again tells you something else.

      If ranks are to be assigned at random, the only distribution function reasonably applicable in your scheme would be to do it via the uniform distribution function. That is to say, each of the qualified student would have exactly the same chance to get selected. Unfortunately, this scheme, unwittingly, would stack things up against the better student, statistically speaking. … Let me explain how.

      Let us consider only the upper—the better—half of the student population. Whether you apply the normal distribution function or some other, all such functions provide a better fit only in the bulky (i.e. middle) range. In statistical modeling, extremities are always problematic. Now, when it comes to measuring conceptual abilities, the distribution—whether the normal or some other—is always going to have, I presume, an ever-falling tail at the higher end. Now assume that there exists a guy who truly deserves to be within the top 0.1 percent. Suppose the band for his case is, say, 3 percent. Precisely because the population count at his true ability level is far smaller than that at the 90th percentile, and precisely because the random draw would select according to the total population, therefore, this guy would fare worse in a uniform draw.

      Now, if you wished to implement a non-uniform draw (a non-uniform distribution function), you would have to conduct a test in addition to having the qualified pool. That defeats the premise.

      I am sure I didn’t succeed in putting the matter across well. There would be some better way to put the same thing I have in mind here. Feel free to drop me an email/post a comment if I am not clear and I would try to explain it once again (after a day or two—so that I can think freshly, from a different angle, about this issue).

      6. Another point. Random draws are not employed if two inventors file the same idea to a patents office on the same day. The patents office people go by the register number then—first come, first served. There is some lesson to be had, I suppose, from this example. In the patents system, we can say that within the limitations of the system, the first to make the claim rightfully gets the patent. The context is important. The patent system is not a means of rewarding merit and talent, actual hard work, or translation of all these into an invention. No, the patents system is not at all a means of handing out rewards to the inventors. The system exists only to let the individual inventor enjoy the benefit of restrictions of entry into the market for a limited time period, to be able to potentially enjoy any profits that may accrue to him—should he care. (Without his invention, the market couldn’t have come into being—hence the monopoly; but continued invention requires that others be able to build on the previous art—hence the limited time duration for the monopoly rights granted.) I have not read a lot on patents and all, and my wording here could be wrong, but the biggest point here is that even in such a great and effective system as patents, we still acknowledge that there are certain limitations to it, just the way there are limitations to any system. Further, in this case, as in many others, we acknowledge that random draws can be arbitrary rather than be fair.

      In the context of examinations, a similar statement is implicit. Within the limitations of this test, we acknowledge that the students scoring high marks might have scored low if they were to take another version of the test. Similarly, a lower-scoring student might have scored a higher score. However, in view of the falling tail (the monotonic decrease at the higher end), since the chance that a lower-scoring student would have scored higher is still comparatively lower than the chance that a higher-scoring student would have scored lower, therefore, within the limitations of this test/criterion, we select the students above the cut-off (whatever it be).

      Note, whether you have quantifiable scores, or papers, or personal interview, any mode would have its own limitations, and that parenthetic statement about there being such limitations would always apply.

      Limitations are not even nearly as harsh as they are made out to be. An inventor may not get a patent if he is the second for no fault of his. Yet, the history of the system shows that it has not failed to reward inventors like Tesla and Edison. Yes, there have been some acrimonious battles/cases, but those are rare and at the highest end of the spectrum. In the average case scenario, why, even software developers working for MNCs in Pune routinely file US patents successfully. And, at the highest end, Tesla could afford a lavish lifestyle. Old wives’ stories apart, if the criteria are objectively right, the attendant limitations do not matter—they are a part of the context.

      7. So, really speaking, the answer to the routine chest-thumping of the JEE IITian (if it can be called that—chest-thumping being too manly) is not, really speaking, random draws. It is: to ask him (and his Professors, Deans and Directors) to each time qualify all of his nonsense by always supplying the appropriate statistical information that qualifies his ridiculous claims.

      8. One final point. Arguments involving comparisons between coached and non-coached students (or between students coming from priviledged and non-priviledged backgrounds) often involve the fallacy of context-dropping… The case is somewhat similar to the idea that socialists like Sitaram Yechuri and Charu Majumdar et al. should have got together and decided in their politburo who would head the Tatas after JRD (or the Ambanis, after Dhirubhai). … Not always, but often enough, that’s the fallacy involved. I mean, there are people who may sound similar and yet actually come from the premise of justice and not of socialism/collectivism, but they are in a negligible minority.

      So bad that (by my own rules) I don’t get to update the comments once I post! [I was trying hard here to recall what is it that they say in English for “chook bhool deNe gheNe,” but, as usual, could not! But you get the idea…]

  2. I will address your reply to my comment later. For now, here’s the equivalent of “chook bhool layni dayni” (as I know it): “Errors and Omissions Excepted” or E&OE.

  3. I believe the statistic about IIT graduates not qualifying in GATE is totally bogus. How many IIT graduates write GATE? Do they have to? IIT graduates with a CGPA of 8 or above are eligible for MHRD scholarships without writing GATE. In fact, the only IIT students who I know of who write GATE are those who have CGPA below 8 and therefore need to write GATE in order to get scholarships. This represents definitely students in the lower half of the class and from doing unscientific surveys, it appears that actually less than 10% of IIT B.Tech. students write GATE. One of the primary uses of GATE is to select candidates for MTech admissions – but IIT BTech students need not go through this process – they can opt to convert their program to dual degree and earn that MTech in just one year extra; why would they write GATE and spend two years instead to get that MTech? Ah, yes, if they dont have 8 CGPA in the BTech, even if they get converted to dual degree, they cannot get scholarship unless they write GATE, so some of these people write and with the aim of just getting a score (just pass will do). About 30% write CAT. The actual number of GATE takers is currently about 3 lakhs and about 30% are declared qualified – so the ‘failure’ rate for GATE takers is about 70%. It is not terribly out of place if the bottom third of IIT graduates take GATE and have a failure rate of 30 or 50%.

    • sriram:

      The statistics about the performance at GATE by JEE IITians is official; it came straight from the IITs themselves a few years ago (not more than five years or so). I had downloaded the results documents (from IIT official servers), and so these documents should be available with me, but I would have to search on my older machine(s)—a task by itself. … But for all you know, the documents may still be available on the ‘net.

      In short, there is nothing “unscientific” about it.

      Re. MHRD scholarships and the five year dual degree program to directly get MTech. Both are indicative of the positive bias towards the JEE IITian that is entrenched in the entire IIT system (and also in the general public outside of them—esp. including the idiots running the US universities).

      Re. CGPA of 8.0. At IITs, the class average is usually recommended to be pegged at 7.0, and due to accumulation over the years, for CGPA, only a few manage to go above 9.5 (sometimes none if a department is difficult, for instance, EE at IITM in the olden days). So, even if the JEE IITians with CGPA < 8.0 were the only ones to write GATE, statistically, they would still come from about 60 to 66% of the population, I guess. Can't call them the "bottom" ones!

      Most important, it is an absurd assumption that only the JEE IITians with CGPA < 8.0 write GATE. This wasn't true 25 years ago, and it isn't true today. Go visit the nearest IIT campus to verify what I say. I personally know of many high-ranking JEE IITians who also did write GATE even back then. In case you don't believe me: the motivation has been to "improve" one's branch by going from, say, chemical to Comp Sci./industrial management. Not that they get it. But they still do try—that's the point.

      It is also very easy to verify that the JEE IITians, as a group, really don't perform all that distinctively but even at GRE. This is interesting because going by your own (false) logic (not new to me), when it comes to GRE, obviously only the better ones would attempt going to the USA, leaving doing MTech at IITs to their inferior class-mates. Now, I have routinely run into the JEE IITians scoring no more than 1250 to 1450 at GRE (V+Q). A significant fraction of the JEE IITians lies in this rather lower range.

      True, that many IITs would have more than one student scoring a perfect 2400/2400 score in the olden days, or today, 1600/1600. One way to explain this is the vocab preparation from the summer vacation of the I year that would go into it. Another is the availability of those hand-written question banks that used to be passed from one batch to the next.

      More important, however, when it comes to IITians as a group, the scores still are distributed over a rather wide band. In other words, *every* JEE IITian can lay the claim to the IIT "brand," but most can't go on to make even a decent 1500/1600 in the V+Q. (If you don't believe me, go ahead, follow the discussions on the threads related to US education at fora like, say, IITIIM.com.)

      Incidentally, the "banding" phenomena is a very general principle, and it applies not only to JEE but also to GRE, GATE, X, XII, open-merit school scholarship examinations, etc. Any examination—you name it. And, if you don't believe me on this count, go ahead, ask anyone who has cracked a 1500+ (or better still, a 1550+) score in V+Q of GRE, whether he would like to rewrite that examination now that he has a high enough score. Or, for that matter, go ask the same question to anyone with an AIR of < 500 at the JEE: would he like to prove his superiority once again.

      Just one more thing. Just remembered it while writing about re-taking competitive exams. I know of only one (actually brilliant) guy who has taken GATE more than once. He is a mechanical engineer, has been a teacher in CS in engg colleges, and, for the fun of it, has appeared in the CS stream of GATE many times by now, some 4 or 5 times—without preparing for a single day at any appearance. His scores have oscillated from late 80s P to 95+ P over different GATEs, in different years (spread over a decade or so). And, I know for a fact that he actually is brilliant (but isn't a JEE IITian). Conclusion: Banding does happen. … So, why rebel against such a massive evidence?

      Nothing of what you argue for is worth it—be it JEE, IIT, or the supposed intellectual superiority of the JEE IITian.

      –Ajit
      [E&OE]

  4. My comment is only about how many IIT undergrads write GATE and how well they score on it. I cannot find any document that has the statistics you claim. Incidentally, the class average is not pegged at 7.0 = C. The class average or 50 percentile, actually, is B = 8.0 in IITM (S=top 5%, A= next 15%, B=next 30% and so on) – so, those getting below 8.0 CGPA would be in the bottom half of the class more or less. Whoever was writing GATE 25 years ago must have been in the first batch, before that there was no GATE. I know, since I wrote the PG admission test about then. The profile of GATE takers has changed significantly since then; in the last 10 years or so (since the dual degree programs came about) it is uncommon to see discipline change at GATE – it is also not possible in many cases since IITs specify both UG major and GATE subject for admission to many branches. Incidentally, I dont have to go very far to visit the nearest IIT campus, since I live and teach in one. I dont know about GRE takers profiles since that is not of interest to me; also, your observation, it appears, is not statistical but anecdotal. And, I dont know what gives you the idea that I argue for some intellectual superiority of JEE IITians. Though I am a JEE IITian myself, I dont believe in any such intellectual superiority crap.

    • 1. About B = 50P. A friend who teaches at IITM told me C = class average, in year 2004. And, C was= class average when I attended IITM in 1985–87. Of course, both are nominal.

      2. The first GATE was in 1983, 27 years ago.

      I know because I was in the final year of BE at COEP; had filled out the GATE form because someone had got an extra; went to the examination hall to write it purely out of fun; and came out of the exam. hall just after about 20 minutes because a friend of mine called me out. (I was seated in the Room 13 of Mech Dept. of COEP then.) This friend of mine was contesting a tightly fought election for one of the top student organization posts at COEP (I forgot which one it was: Regatta Secretary or Boat Club General Secretary). He needed help in canvassing, so called me out. I hardly cared for GATE and so turned the paper in and went out. (The friend lost that election by exactly 1 vote after a record voter turnout of 1200+ students out of about 1600.)

      When I wrote my next GATE in 1985, once again, I was “equally” well prepared—meaning, no preparation whatsoever and a late night beer party with friends on the night before the examination. Now that is what 25 years mean!

      Now these are what I call anecdotes.

      3. CS still remains open, does it not? Also IT? Industrial Engineering? Management?

      4. Which observation of mine is anecdotal and not statistical?

      5. About the absence of the JEE superiority syndrome in your case: Good for you!

      Drop me an email; I am curious as to who you are.

      –Ajit

Comments are closed.