MIS = Too many mistakes!

There are three kinds of lies: lies, damned lies, and statistics. And the last one seems to hold true for the Management Information System (MIS). Once praised by development thinkers for making data transparent and available in the public domain to be equally accessed by all, thus ensuring Section 4 (1) of the Right to Information (2005) Act, the MIS for monitoring NREGA at the Gram Panchayat (GP) level has come under intense criticism from the Comptroller and Auditor General (CAG) audit report number 6 of 2013.

The CAG report on the Performance Audit of Mahatma Gandhi National Rural Employment Guarantee Scheme (MGNREGS) came down heavily on the MIS of the job scheme for irregular data entry made in various states and errors in the database. The CAG found the MIS data on physical and financial performance of NREGS to be unreliable.

Pic: http://nrega.nic.in

In its Second Performance Audit of MGNREGS submitted to the Parliament on 23 April, 2013, the CAG discussed large discrepancies between the data uploaded in the MIS and the actual paper records maintained or information available with the Department/ District Project Coordinator (DPC). The Government’s e-governance effort for transparency and efficiency concerning MGNREGS is at stake since discrepancies in data were observed in all types of records including number of households registered, number of job cards issued, job card number, employment demanded, employment provided, number of works, expenditure etc. Therefore, the question that can be evoked is whether the MIS is capable of real time monitoring of the MGNREGS.

Noticing unsatisfactory monitoring at the Central level, the Audit found that the Central Council did not fulfill its statutory mandate of establishing a central evaluation and monitoring system, even after six years of its existence. Deficiencies, relating to both non-maintenance and incorrect maintenance of prescribed basic records, were noticed in 18 to 54 per cent of all test checked gram panchayats, for various types of records.

In its Performance Audit Report Number XXXX11 of 2008 of the NREGA, the CAG noted that the absence of Gram Rozgar Sewaks adversely affected the maintenance of basic records at the gram panchayat level, without which it was impossible to verify employment demand and allocation for each household.

CAG report: Problems in the MIS

Some of the discrepancies observed by the present CAG Audit are as follows:

• In Assam, Goa, Gujarat, Jharkhand, Nagaland, Punjab, Rajasthan and Lakshadweep, job card numbers of beneficiaries entered into the MIS did not match actual records. For example, data made available to the Audit revealed that the number of job cards issued in four blocks of Kamrup district of Assam stood at 0.84 lakhs, whereas the same stood at 0.90 lakhs as per the data uploaded in the MIS.
• The position of fund balances entered in the MIS did not tally with those entered in the bank passbook and other basic records in the case of Andhra Pradesh, Goa, Kerala, Maharashtra and Nagaland.
• The employment generation figures entered in the MIS did not match those given in the actual records in the case of Bihar, Goa, Gujarat, Jharkhand, Kerala, Punjab, Rajasthan, Tamil Nadu, Uttarakhand and Puducherry.
• There was a variation in the expenditure figures in the MIS and those which were available in the Monthly Progress Reports (MPRs) in Assam, Bihar, Goa, Gujarat, Jharkhand, Punjab, Uttar Pradesh and Lakshadweep.
• In Arunachal Pradesh, Bihar, Gujarat, Jharkhand, Jammu & Kashmir, Mizoram, Maharashtra, Odisha, Rajasthan, Sikkim, Uttarakhand and Dadra & Nagar Haveli, data was not entered on a regular basis.
• In Goa, Gujarat, Haryana, Jharkhand, Kerala, Madhya Pradesh, Maharashtra, Odisha, Punjab, Rajasthan, West Bengal and Puducherry, the authenticity and correctness of data that were fed into the MIS was not tested by cross-checking it with original records, and therefore likelihood of errors remained high.
• Due to faulty programming logic and missing validation controls, the MIS has suffered. The CAG audit found cases of data manipulation, without any reference to basic records. Data fudging without any apparent basis was also quite common.

About NREGAsoft

The Ministry of Rural Development (MoRD) along with National Informatics Centre (NIC) and other stakeholders, conceptualized the web-enabled MIS-NREGASoft, so as to address the planning & monitoring needs of the MGNREGS. NREGAsoft is a local language enabled, work flow based, e-Governance system and is available in offline as well as online mode to capture all the activities under NREGA at the Central, State, District, Block and Panchayat levels, according to the 2012-13 Annual Report of MoRD. NREGAsoft was prepared and deployed at http://www.nrega.nic.in by NIC in collaboration with MoRD.

Some of the objectives of NREGAsoft as retrieved from the MIS website are as follows:

• Bring transparency to the entire system
• Make various registers, muster rolls, documents available to public
• Provide single window interface for all stakeholders of NREGA
• Maintain records of 100 days of employment to a family
• Maintain accounts and generate all registers/documents to be kept at Gram Panchayat in the format specified in guidelines
• Track transfer of funds to various implementing agencies
• Decide when and how much funds to replenish in which account
• Highlight the irregularities, send alerts to various stakeholders

CAG report: Information Technology (IT) Audit of NREGAsoft

An IT audit of NREGAsoft by the CAG revealed the following flaws:

• Close scrutiny of data relating to work in progress by the Audit exposed that in 56,24,004 records in 20 states, certain columns (i.e. "Entry by" columns in all major tables and "Authorized by" columns in Registrations and Applicants Tables) were either left blank or had been filled with ambiguous data such as "Guess", "Test", "Computer IP Address", numbers or single/ double alphabets, etc. Thus, it was impossible to identify users and to trace back transactions to the originator. The NREGAsoft did not have a system to uniquely identify all users and ensure that data entry was permitted only to authorised users. The accountability mechanism for preventing erroneous data entry was missing.
• Operational Guidelines were violated by the MoRD so that person/s responsible for preparing and scrutinising data could not be identified. Provision of IP address tracking could help in getting the location of data entry but was of no use to identify users.
• NREGAsoft accepted invalid and incomplete information without generating alerts on occurrence of such an event. For example:
• Ambiguous/ invalid names (containing either numbers or special characters) of registered beneficiaries were found in 1,23,849 records in 18 states.
• The Audit found missing/ invalid house numbers of registered households in 6,42,14,836 records in 19 states.
• Data entry personnel were not aware of the working of the programme and were not conscious of the impact of common entry errors.
• Comparison of the MIS data provided by the MoRD with that available at the State Rural Development Departments of the state governments of Karnataka and Odisha, revealed unauthorised instances of deletion of data and revision of values in certain cases.

Why is MIS preferred in India?

The web-based MIS places all critical parameters such as shelf of projects, sanctioned works, wage payments, number of days of employment provided and works under execution online for easy public access. MoRD’s MGNREGA Sameeksha (2012) report discussed that the MIS captured all information on the beneficiaries (including name, photograph, house location, etc.), works (attendance of beneficiaries, location of work, person-days generated, etc.), expenditure (amount spent on material, amount spent per work, etc.) and other processes and procedures (delays in registration, wage payment etc.). Thus, MIS which provides real time online data was viewed as ideal for the purpose of monitoring because of its reliability.

According to the MGNREGA Sameeksha (2012) report, the Monthly Progress Reports or MPRs were mostly hard copies of progress reports passed from the gram panchayats to the block and then to the district, to be entered online. MIS was preferred over paper-based MPRs since the latter could not be drilled down beyond the district level, which made it difficult to monitor Gram Panchayat-wise performance, issues and expenditure during 2006-07. For FY 2011-12, more than 90 per cent of the total expenditure is available on MIS (including details of beneficiaries etc.).

MGNREGA Sameeksha: Differences between MIS and NSS data

(click on image for larger view)

The MGNREGA Sameeksha report disclosed several discrepancies in the data reported by MIS and the National Sample Survey-NSS 66th Round data [July 2009-June 2010].

Table 6.4 titled "Difference in NSSO 66th Round and MIS/ Monthly Progress Report Data" (click here) of the MGNREGA Sameeksha (2012) report revealed that

• As per the MIS and MPR data [July 2009-June 2010], the percentage of rural households in India that were provided employment under MGNREGS stood at 25 per cent whereas the same as per the NSS data [July 2009-June 2010] was 24 per cent. The differences are starker between MIS and NSS data on the same for states like Andhra Pradesh (MIS/MPR: 50; NSS: 35) and Jharkhand (MIS/MPR: 44; NSS: 16).
• As per the MIS and MPR data [July 2009-June 2010], average person days of work per household under the MGNREGS in India stood at 54 days whereas the same as per the NSS data [July 2009-June 2010] was 37 days. The differences are starker between MIS and NSS data on the same for states like Andhra Pradesh (MIS/MPR: 61; NSS: 46) and Himachal Pradesh (MIS/MPR: 64; NSS: 47).

As per the NSS 66th Round data, only 35 per cent of the total rural households had job cards. But as per the MIS, this number is around 53 per cent (of the total households).

Clearly, MGNREGA Sameeksha (2012) report made no distinction between MIS and MPR data assuming that all MPR data were correctly digitized into MIS. The same report emphasized that "data on the MIS/MPR is real time, whereas NSSO data is survey data". But the CAG Audit report found discrepancies between the MIS and MPR data as well.

Why do MIS and NSS data differ?

Experts opine that discrepancies take place due to misreporting or tampering with the data. Discrepancies were usually higher in states/UTs with higher corruption in MGNREGA, lower awareness and weaker accountability systems.

By comparing data from the NSS 66th Round and administrative statistics of the MIS, Yoshifumi Usami and Vikas Rawal, in their article titled Some Aspects of the Implementation of India's Employment Guarantee (2012), also found that there was a gap between the number of households that were reported to have job cards and the total number of job cards that were issued.

According to the NSS data, 56.5 million households had job cards, whereas the MIS data showed that a total of 116.4 million job cards had been issued until 2009-10 (see Table 4 titled “Number of rural households with job cards, National Sample Survey and Management Information System of the Ministry of Rural Development, 2009-10”). Since the NSS used a different definition of household as compared to MoRD's NREGA, there was every incentive for beneficiary households to sub-divide themselves and to apply for separate job cards for each family in case of the latter.

However, Usami and Rawal (2012) proved that the discrepancy between the number of households with job cards and the number of job cards issued happened also because the MIS data came from administrative records that were manipulated by local-level administrators. There was a strong possibility that a number of cards were issued officially, but not provided to the households. This was corroborated by their rigorous empirical study which found that as per the MIS data, a substantial proportion of households that officially held job-cards never sought work or worked under the Scheme.

It was also found that the MIS data over-recorded the level of employment provided to households. The total number of person days of employment recorded by the administrative (MIS) records is nearly 22 per cent higher than the person days of employment reported by households in the NSS.

(click on image for larger view)

Data from the 66th Round of NSS Survey on Employment and Unemployment also indicated that a substantial proportion of rural households demanded work under MGNREGS, but were not provided any work at all, as recorded by Usami and Rawal. About two-thirds of such households were not given work because they had not been issued job cards. The difference in the MIS data between the number of persons/households who demanded work and the number of persons/ households who were provided employment was very small. The MGNREGA Sameeksha (2012) report too argued that in case of MIS data, one could see marginal differences in number of households who demanded work and those who received work. However, the NSSO data may not be entirely accurate for the same since it is based on a 365-day recall of the beneficiaries. It may not be possible for the beneficiaries to be accurate in recalling information over a period of one year whether they sought work under the job scheme during the last 365 days.

What is to be done?

The rampant discrepancies noted naturally raise questions as to the way forward regarding accurate data reporting with respect to social welfare schemes such as the MGNREGS. On one hand, technology is not a silver bullet for effective implementation and monitoring of MGNREGS. But on the other hand rejecting the MIS outright would be like throwing out the baby with the bathwater. The system should have proper checks and balances. Keeping that in mind, the CAG made certain key recommendations as follows:

• Record maintenance at the Gram Panchayat level requires to be streamlined. This should be monitored closely at all levels and funds should be linked to proper maintenance of records.
• Original data fed into the system should be checked before updating the MIS.
• There is a need to put in place stricter controls for data modification after authentication and closure of data entry.
• The staff deployed to feed the data should be properly trained and they should be held accountable in case of data fudging. Operational Guidelines are required to be followed.
• Lessons can be emulated from Andhra Pradesh's e-Muster, e-Measurement, e-Muster verification and e-check Measurement.

The MGNREGA Sameeksha (2012) report acknowledged the problems with the MIS and suggested that more national level survey-based studies are required to verify the authenticity of MIS data.