Scorecards: A General Overiew

…too many financial institutions and investors simply outsourced their risk management. Rather than undertake their own analysis, they relied on the rating agencies to do the essential work of risk analysis for them.

Lloyd Blankfein, CEO Goldman Sachs

Credit risk scoring is simply a statistical tool for evaluating the level of credit risk associated with applicants. A scorecard is like a table. If the applicant hits certain features/attributes, certain points will be assigned. If the total points passed the cutoff point, the application is successful.

The statement is simple enough. But as an intelligent reader, you should have a few questions in mind. For example, what do you mean by credit risk? It is quite vague and hard to measure. If it is hard to measure, how can you evaluate risk level using statistical methods (since any scientific method requires measurable quantities)? How can one build the tool? What is the process, etc? In this article, I am going to answer those questions and give you an overview on what is a scorecard.

Defining Credit Risk

Defining a measurable quantity for credit risk is very important step (but not the first step) in developing credit risk scorecard. It is our target variable. The thing we want to evaluate.

Traditionally, credit risk is a binary target variable, Good/Bad. Bad is generally defined using negative performance indicators such as bankruptcy, fraud, delinquency, write-off/charge-off, and negative present value. The selection of bad definition is both a business and a statistical problem.

The business (as a company) problem is the trade-off between revenue and loss. If it is too strict, more applicants are rejected. On the other hand, if it is too loose, credit loss will be bigger. The statistical problem is that if the bad to good ratio is too low (which is good for the company), the scorecard developed will be biased towards “Good”.

Having the target variable is not enough, we need something to predict the target variable: features.

What features should be included in the scorecard?

The features included should be similar to what a credit analyst would look for in approving new applications. In reality, this is challenging because of feature measurability and data availability.

Not all characteristics in which credit analyst look at are measurable. Some factors are more qualitative, such as macroeconomics situation, competitiveness within the industry. Although it is my belief that qualitative features can ultimately be quantifiable, it requires much more data to do so, for example, we can quantify macroeconomics by using more parameters.

Data availability is another big challenge which is often overlooked. Very often, the features we want to include are just not available. It is because data is not stored in database. This is understandable since credit analysis is evolving and the original design of database may not consider the future use cases.

Apart from data availability for past data, we also need to consider data availability in the future. If it is not available, the scorecard is of no use.

Gathering data that is not stored in database is a time consuming and labor intensive task. Certain trade off between number of features and data gathering should be made.

Overview of the process

Now, we have gone through the two important steps of building a risk scorecard. In this section, I am going to show you the bigger picture. It is ok that you don’t understand a few jargons. The idea is to get the big picture.

Tip: Don’t jump right into scorecard development or Definition of bad and feature selection!

Stage 1: Planning

• What you want to achieve by scorecard?
• Internal development or vendor solution?
• Project plan and project team

Stage 2: Data Review and Project Parameters

• Data availability and quality assessment
• Data gathering and definition of project parameters
  • Definition of bad
  • Features selection

Stage 3: Development Database Creation

• Store data for developing the scorecard

Stage 4: Scorecard Development

• Data exploration: missing data, outliers, correlation, distribution
• Binning
• Preliminary scorecard
• Reject inference
• Final scorecard
• Validation

Stage 5: Post Development

• Monitoring
• Reporting

I hope the article can give you a big picture of what is a scorecard and the development process. More importantly, we need to be aware of the limitations and trade-offs in scorecard development.