Comments on MHRD document on JEE

This blog is a combined effort of members of Faculty body at IIT Delhi

Summary of Comments and Resolution on The Draft Document on Joint
Entrance Examination for Admission to Engineering Programmes in CFTIs
(JEE)

Faculty Forum, IIT Delhi
(Meeting held on 9th April 2012)

The faculty body noted that as per the IIT Act, admission to IITs and
the conduct of the entrance examinations come under the purview of the
Senates. It is recognition, by the framers of the Act, of the fact that
the domain experts on matters of academic quality are the faculty bodies
of the IITs.

However, since the Director, IIT Delhi, has circulated the Draft
Document on Joint Entrance Examination for admission to Engineering
programmes in CFTIs (JEE) to all faculty for comments, the members
discussed it in detail.

The faculty members unanimously resolved that the proposals in the
Draft Document on Joint Entrance Examination for admission to
Engineering programmes in CFTIs (JEE), as they stand, are seriously
flawed and will result in severe distortion of selection processes and
educational outcomes of the concerned institutions.
The stated objectives of the proposed reform are (i) to reduce the
stress, on students, of appearing for a multiplicity of entrance exams
(ii) To enhance the importance of the class XII board examinations and
(iii) to reduce the influence and growth of coaching institutions. We
believe that the proposals in the draft will not achieve any of the
stated outcomes. Objective (i) is based on a flawed assumption that a
single exam, likely to be interpreted as one “do or die” chance by
students, will reduce stress. In order to address the second objective,
the proposal envisages giving 40% weightage to board exam results. This
has several serious shortcomings, as discussed in detail in the
accompanying document; we believe that without due attention to
statistical design and implementation details, this proposal if carried
out would result in gross distortions in the selection process.
Objective (iii) will not be achieved as a consequence of the proposed
reforms. On the contrary, we expect that there will be an expansion in
coaching institutes, targeting the board examinations and the new format
of the proposed entrance tests.

Such sweeping and fundamental changes, as proposed in the Draft
Document, should be undertaken only after detailed and informed analysis
of various options. It is tragic that there is no evidence of analysis
of any depth in the proposal. The undue haste with which matters of
national importance are being addressed and the accompanying accelerated
time schedule leave a lot to be desired.

The Faculty body is of the view that any change in the existing JEE
format should be arrived at only following a detailed and systematic
academic exercise. The starting point of such an exercise could be
workshops conducted in each of the IITs in which the entire faculty body
would be involved, along with representatives from other stakeholders.
This could be followed by discussions in the respective Senates and
Pan-IIT meetings to evolve a mutually acceptable and academically sound
proposal.

******

Resolution of the Faculty Forum, IIT Delhi

(Meeting held on 9th April 2012)

The faculty members unanimously resolved that the proposals in the
Draft Document on Joint Entrance Examination (JEE) for Admission to
Engineering Programmes in CFTIs, threaten to undermine the academic
autonomy of the IIT system to select appropriate students for its
undergraduate programmes.

As per article 28a of the IIT Act, the admission of students to IITs is
governed by the Ordinances, which are the prerogative of the respective
Senates (article 29 (1) of the IIT Act.) Further, as per Article 33
(2)a, the role of the IIT Council vis-à-vis admission and other academic
matters is purely advisory. In the light of this, the proposals in the
Draft Document must be treated merely as advisory inputs to be
considered by the Senates as far as IITs are concerned.

However, since the Director, IIT Delhi, has circulated the Draft
Document to all faculty for comments, the members discussed it in detail
and resolved the following:

Objectives:

As per the draft document (and the Ramasami Committee Report) the
objectives of the proposed reform are:

1) To reduce the stress, on students, of appearing for a multiplicity
of entrance examinations.
2) To enhance the importance of the class XII board examinations.
3) To reduce the influence and growth of coaching institutions.

It is quite obvious that these objectives are not met by the proposed
changes.
i) A single examination combining IIT JEE and AIEEE could in fact lead
to larger stress levels as the candidates get in effect a single
opportunity and one bad day could cost them a full year. Having the
examination twice a year does not prevent this as the academic session
starts only once a year. There is the added complexity of comparing the
scores obtained in the two examinations. This has not been adequately
addressed in the proposal.
ii) As per the survey conducted by the Ramasami Committee, at present,
students appear for at least five different entrance tests. The
proposed change would at best reduce this by one.
iii) To ensure that class XII examinations are taken seriously it is
enough to include them in the screening process. There is no need to
include these scores in determining the final rank.
iv) Additionally, should not the emphasis really be on higher secondary
education rather than on the class XII board examination? Such a
discussion was part of the Acharya Committee Report (Section 11). It is
rather unfortunate that subsequent Committees have focused merely on
class XII examination rather than school education.
v) The enhanced focus on class XII examination and possible
psychometric tests would in fact greatly exacerbate the influence of
coaching institutions rather than curtailing them. This is evident from
recent newspaper advertisements by some coaching institutions.

However, all this is beside the point. Surely the primary objective of
JEE must be to select the most Competent and most Committed students
with an Aptitude for Engineering!! This objective deserves to be
explicitly highlighted if the selection mechanisms are to be
appropriately designed. It is notable that it finds no mention in the
entire Ramasami Committee Report or the subsequent draft document.

Major Technical Flaws:

1. The main thrust of the proposal is the inclusion of class XII board
performance in arriving at the final rank. The weightage given to this
would be at least 40%. There are several flaws with this:
a. In the Ramasami Committee Report, the assumption underlying the use
of percentile scores is that the top bracket of students in all boards
has identical abilities. While this may be the best assumption in the
absence of any data, it can by no means be taken as correct unless
independently substantiated. Moreover, within each board, we would
expect the high-scoring end of the distribution to be subject to
considerable random error.
b. Use of percentile scores would entail a bias towards providing
quotas to each of the higher secondary boards, without any consideration
given to the relative academic quality of the boards.
c. Any normalization between boards would have to be validated on the
basis of a common examination. There is no such exercise envisaged in
the proposal.
d. Apprehensions have also been expressed about the way Board
examinations are conducted and evaluated. The incidence of cheating in
some centres only adds to the complications of arriving at a fair
normalization scheme.
e. The proposal is to include the sum of board marks from Physics,
Chemistry, Mathematics, one language and one unspecified elective. This
is not academically sound since the different subject components are of
variable relevance in determining fitness for engineering/science, and
the unspecified elective could prove to be another randomizing factor.

2. The statistical distributions of the three components of the
proposed JEE score (namely the percentile score based on the Board
examinations, the Main test and the Advanced test) are expected to be
widely different. It is not clear, a priori, how the weighted sum of the
three would behave. A very careful sensitivity analysis needs to be
conducted before such a composite score can be accepted as the basis for
ranking.

3. Further, the purpose of the Board examination and Main test should
be to act as a coarse filter. On the other hand, the goal of the
Advanced test is to act as a fine filter for the CFTIs. It is only
logical that the result of the first be used to shortlist candidates for
the Advanced test. Employing a coarse and fine filter on the same
population is futile.

4. The use of psychometric tests needs to be carefully considered in
view of widespread criticism of their ability to assess personality or
intelligence.

5. The proposal of specifying the percentile score to “9 decimal
places” is incorrect. It is mathematically impossible to resolve scores
to this resolution since the total marks cannot have a resolution better
than one in 500 (since 5 papers, each evaluated out of 100 marks, are to
be included as per the proposal).

6. Any major changes, such as those proposed, should be implemented
only after thorough dry runs across all boards. It has to be kept in
mind that any failure could be disastrous.

7. Any major changes, such as those proposed, should be implemented
with a lead time of at least two years (that is after 2014). Parents of
many aspirants would have invested their life savings in helping their
wards prepare for entrance examinations. It would be grossly unfair to
thrust sudden changes, of the type proposed, on them. Further, the
students appearing for class XII in 2012 are not aware of the proposed
policy of including class XII marks in the determination of JEE ranks.
Many of them would be planning to appear for entrance examinations in
2013 also.

Additional Points

1. Intake into the IIT System and the conduct of the entrance
examination is under purview of the Senates of the IITs. To date, the
proposed changes (Ramasami Committee Report) and the time schedule have
been rejected, after careful scrutiny, by Senates of all those IITs that
have discussed the matter (IITD, IITB, IITK).

2. IIT faculty members who have taught students prior to 2003 have all
independently noted a considerable drop in the quality and aptitude of
students after the detailed pen and paper component of JEE was
eliminated. This point has come up in several internal academic
discussions.
The proposed changes in the Draft Document will render this drop in
quality even more deplorable.

In view of this the IITD Senate has recommended that JEE revert back to
the 2-stage format with the second stage being a pen and paper type
(non-machine readable) examination to be conducted exclusively by the
IITs. This was also recommended by the Acharya Committee. It is not
clear why the Ramasami Committee (which was set up presumably to evolve
the modalities of including Class XII results in the screening stage of
JEE) and subsequently the IIT Council chose to ignore this
recommendation.

3. Coordination between several dozen boards, to be ensured by JIG, as
per the Draft Document is likely to result in an operational nightmare.
Any delay in the announcement of the results of even a single board
could potentially derail the entire admission process.

4. Such sweeping and fundamental changes, as proposed in the Draft
Document, should be undertaken only after detailed and informed analysis
of various options. It is tragic that there is no evidence of analysis
of any depth in the proposal. The undue haste with which matters of
national importance are being addressed leaves a lot to be desired and
raises questions on the motivating forces behind these proposals.



Proposed Approach

The Faculty body does not wish to imply, through this analysis, that
the existing JEE system is either fool-proof or perfect. However, any
change in the existing JEE format should be arrived at only following a
systematic exercise such as:


a. A Two-Day Workshop (in say the 2nd week of May 2012) in which the
entire faculty body of each IIT would engage in deep introspection and
detailed discussion of the JEE process with the primary goal as stated
earlier. Representatives of MHRD and various Higher Secondary boards
would be invited to participate in this workshop (as also other
stakeholders).

b. This would be followed by Senate meetings and a Pan-IIT meeting to
decide on a sound and mutually acceptable format for JEE.
Representatives of MHRD and various Higher Secondary boards would be
invited to participate in this Pan-IIT meeting also.

Prepared by:

C. Chakravarty, A Chawla, S Kohli, M R Ravi, S Sanghi, S V Veeravalli