How does ICAI evaluate answer papers? How are marks allotted, especially in theory papers?
ICAI evaluation procedure
Over the past 1 year, I have been helping students as a Counsellor in making career decisions & improving study habits to succeed in exams under my EduSaarthi initiative. In this process, I have reviewed papers of hundreds of students, to help them identify their real mistakes, to know what is wrong with their study habits and which areas need more attention. Further, to enhance the quality of advice, I have also contacted many examiners, to discuss how papers are evaluated and what is expected from the student. Having the privilege to interact with ICAI Presidents, Chief officials & Members of Board of Studies on various occasions in past, I have discussed the issues relating to Paper Evaluation & obtained an understanding of the Institute’s perspective on the same. I have also reviewed certified copies of Merit Rankers to note key differences that they did which students at large failed to do.
So with that background, the following answer is no prediction or guess work but completely based on the knowledge obtained in this process. It is illegal to publicly share the certified copies or the details of examiners and hence, I would only share the experience gathered from the same.
Selection of Examiners –
- Theoretically, there are three levels of evaluators – Examiner, Moderator and Chief Moderator, however, it is only the Examiner who evaluates your paper in detail, while the Moderator quickly reviews your papers and Chief Moderators only review certain sample papers for assessment of quality.
- All examiners are selected on the basis of qualification and minimum experience. For subjects like ISCA and IT, institute also hires the services of University Professors who are experts in the subject. Most examiners are practising Chartered Accountants.
- There are also trainings & mock paper evaluations conducted to assess the examiners. While evaluating papers after selection, if there are multiple instances where moderators find excessive or less marks allotted for an answer, such examiners are expelled and barred.
Process of Evaluating Papers –
Following is the process followed today. Past is history and let it rest in peace.
- ICAI provides an answer key to all examiners. The answer key contains all the answers, all alternate answers, marks to be allotted for each step and it also contains notes for the examiners which basically guides them to give marks for variation in answers or to allow answers which virtually means the same.
- If there are suggestions received from the student about difficulty faced in question paper or multiple answers to a question, or if the same is noticed by members or examination board, supplementary instructions are issued to the examiners. If it is too late to issue instructions to rectify the marks, moderators do the same, when they get the papers.
- To maintain integrity in the process of evaluating papers, examiners only allot marks and the task of totalling the marks in done by a separate person who tallies the paper (because examiners are humans & emotional humans may give extra marks if they know someone has scored 39 or give fewer marks if they know someone has scored 41.)
- After evaluation by examiners and totalling by another person, papers reach moderators who simply review the papers. While reviewing papers, if they notice some examiner has not done his job correctly, complaint against examiner is raised and all papers are revaluated by another examiner or moderator if found necessary.
- The evaluation done by Moderators is a grey area. Many popular beliefs & rumours have developed only because this process isn’t as transparent as it should be. As per my experience, moderators are the ones who at times grant grace marks to the students if they are around the cutoff; they also grant marks when there are mistakes in question and students must be compensated for the same; they also deduct marks of students at large for answers which are controversial but ICAI has its own wierd point of view.
How sums are evaluated?
- Sums are evaluated entirely on a stepwise basis. There are marks for certain steps which are important according to the relevant committee of the examination board. If the particular step is missing or calculated wrong or skipped, marks for that step won’t be granted.
- Here step includes every calculation included to reach its final answer i.e the formula, the values & then the solution of that step. If all three are correct only then the marks are allotted for that particular step, else partial or zero marks are given if even a single element is missing.
- Notes are important & there are marks for workings as well. Even if your entire answer is correct but if workings are not properly explained, the examiner may deduct marks or even refrain from giving answers, in cases where the working was the heart of the answer and student completely skipped it. This happens irrespective of the fact whether your answer was entirely correct or not.
- In subjects like Costing, the examiners are very strict because this subject is entirely based on working. All data is given in sums, all you have to do is work out a solution with them. Therefore, if you fail to explain your answer properly, the examiner will not grant you marks.
- So those calculations you did in your mind, or on the back pages of answer sheet & the question paper, or the ones that stay only on the calculators – they all need to be jotted down on the answer sheet. So don’t keep workings to yourself, share it with that poor guy who is going to evaluate the paper.
How theory questions are evaluated?
- Due to controversies in the past, ICAI has now adopted a strict “Evaluation as per answer key’ policy. If your points are same as those mentioned in answer key, you will be granted marks.
- There is a step wise marking in theory questions as well. Certain important points carry more marks sometimes, while other points may carry less marks.
- A four marks question may expect four points & explanation or eight one liner points depending upon the explanation given in the ICAI Study Material.
- This doesn’t mean your answer has to be word to word same as in Study material (many coaching classes & faculties have explained this wrong). It is important that your answer contains the same points that the examiner expects. The way you explain the point can be different and there is no issue if you use simple language except for the specific terminologies.
- If you answer gives the same rationale as given in model answers, you will be given more than 60% marks in that answer. Sometimes, examiners also give full marks, if your answer covers all the points necessary and explain the point appropriately.
- There are at times when answer only expects 7–8 one liner points while students go on answering it long & with an explanation. The trick to identifying this allocation is knowing what is the answer given in the Study Material. So it is important to KYSM – Know your Study Material.
- Introductions, diagrams, flowcharts, borders, pencil work, lengthy answers, etc. can be saved for Bcom exams. In this course, the examiner doesn’t pay hid to them as there are no marks for the same. Remember there is a stepwise marking for all the answers.
- This is evident by the fact that nowadays examiners even underline/circle specific parts of the answers and then allot marks in the left margin because they are hinting their moderators why they gave the student marks.
- Usage of certain technical terms like Professional Skepticism, Assertions, Pervasive, Prudence, etc. is important to score better. These words should not be replaced with other words. To score more than 60, you must use appropriate technical terms pertaining to our profession. It will be very helpful later in the Corporate world as well.
- Toppers are not doing anything special, not all of them have great handwriting or genius command over memory or language. They are just doing fine like every other student. The difference is, they don’t commit mistakes, their answer is precise, they know what question expects and they write exactly the same.”
Issue is the system, not evaluation –
- If the exam is subjective, how can the papers be evaluated objectively? 100% objectivity is not possible. That’s why ICAI has answer key to question papers which mentions all alternate answers and asks the examiner to give marks only if the answer matches with the answer key. This eliminates subjectivity to an extent, however, subjectivity in evaluating whether the explanation to your answer is appropriate or not, still remains and this can never be eliminated, unless the Institute opts for multiple choice examinations.
- Teachers exist to fail you, that’s what we have been believing since childhood. In school or college, we never had issues about subjectivity because there was only one or two teachers or professors who evaluated the papers of entire class and therefore, evaluation remained same for all students. However, when it came down to 10th or 12th Board exams, similar concerns are raised, questioning if examiner is being objective enough while evaluating papers. Unlike school days, in CA course, not all papers are evaluated by the same examiner and therefore, the paper checking may involve 5–10% variation, since so many experts are involved.
- Institute believes “Subjective exams are better. Outcome of CPT exams justifies it.” Institute has made several attempts to improve the evaluation, however, it reluctant to change the examination system. The institute believes subjective form better than objective questions and CPT is also one of the reason behind this belief. Therefore, institute is constantly improving the evaluation process and I do believe, there is a good progress in the same. However, I also believe, as long as subjective form of examination exist, this issue is going to stay.
- Saurabh Toshniwal believes “Reduce the number of students in the pipeline, increase the fees if revenue is the concern.” The number of students who pass Foundation exam is high, followed by fewer number of Intermediate students and even less Final students. More students, means more examiners and more examiners means less internal control, more subjectivity. As we learn in Audit, internal control in a closely held company is better than in larger organizations, as it involves fewer people & issue can arise only if owners override the controls.
Conclusion: As long as, examinations are subjective, there will be subjectivity in paper evaluation as well. The institute believes in subjective form of examination and therefore, the subjectivity in evaluation will also accompany.
Note: I hope you know, I am not the one who set this process. So, if you didn’t like the process, you know whom to disturb, message, complaint or argue with. However, you may ask related queries and avail the benefit of my experience.
Queries raised by Readers –
I am thankful to the readers for raising queries. It helps in resolving the expected concerns of other readers as well.
 “Does ICAI really do Scaling i.e. deduct xx marks from all the papers to maintain the results?” asks Rahul
Yes and no both. It does because results & public distrust cannot be fake and it doesn’t because rumours are spicier than reality. As I mentioned, Moderation is a grey area and is the root to all issues. Probably moderators are given instructions to check the papers easier or stricter. There can also be specific instructions about specific questions which were either too difficult and hence grant grace marks or controversial where Institute is not ready to budge.
Rumours say, “ICAI does scaling and deducts 10 marks ad-hoc whenever and whichever paper it wants to. It sees the totals & reduces marks from whichever paper it wants to. Institute is a moody kid.”
Reality is, “Nobody knows what happens, as Moderation is a grey area. The only fact public is, strict-checking is done subjectwise as moderators are not the same for all subjects and therefore, nobody knows the marks of your other subjects or the total marks of your group. Institute is a moody kid and surely strict-checking is also unfair.”
 “If the question paper is easy will the paper checking be strict and vice versa?” asks Rahul
Yes, but I will rewrite the statement as, “If question paper is very easy then the evaluation will be very strict and if question paper or a certain question is very difficult, evaluation will be easy.” This is evident from the results in the past.
 “Does institute provide marks for format and presentation, even if answer is incorrect?” asks Sumit
No. If answer is wrong, you don’t get marks, except for the steps & adjustments which were right. If your answer is right, but your presentation or format isn’t up to the mark, you will still get full marks for the answer, unless you are amongst the toppers. Remember, evaluation is stepwise. If your steps are correct (irrespective of formatting), you will get marks.
 “When you say step wise marking does it mean if at certain step answer is incorrect no marks will be provided after that step?” asks Sumit
Generally yes, if the question is entirely linked to that value, all the steps after the incorrect step would be calculated incorrectly and therefore, the examiner won’t give you marks. However, sometimes there are important adjustments in the answer, which are totally unrelated to the wrong step, in such case, you will get marks for the other correct adjustments as well.
E.g. In a Cash Budget question, if there are adjustments relating to Debtors, Creditors, Purchases, Expenses and rest all is addition or substraction, then 2 marks will be for Debtors adjustment, 2 marks for Creditors, 1.5 mark for purchases, 1 marks for expenses, 1.5 marks for final answer, if the answer is for 8 marks. Now if your Creditors adjustment alone is wrong, but everything else is correct, then you will get 2+1.5+1 = 4.5 marks as the creditors working (2 marks) & final answer (1.5 marks) wasn’t right.
 Are there internally decided cut-off marks for each subject to declare a students pass? asks Anuj
No. The only cut-off for each subject to declare a student pass is the Passing Marks cutoff i.e. 40 marks in a subject. There is no other cutoff decided.
 “Some students prepare for Advanced Accounting & Auditing in detail, but ignore ITSM paper. They get 60–75 in Advanced Accounting & Auditing and in ITSM they get exactly 40 marks.” asks Peratchi
These are our oversmart students. After the extremely poor results in subjects like ISCA and ITSM, institute eased the ITSM papers (made it Bcom style) and also resorted to simple evaluation of paper, not expecting detailed use of technical terminologies. So these oversmart students, abuse this very mechanism to pass exams. Do not opt for any such strategy, examinations are just means, knowledge is the ends
Credits: Saurabh Toshniwal, CMA, CA, Works at E&Y, Ex-Deloitte, Mentor, Speaker, Writer