What is a policy, from a retail finance credit risk perspective?

In my current company, Seamoney, there are two major teams responsible for managing the credit risk of the consumer loan portfolio: the Data Science Team (DS team) and the Credit Policy Team (Policy Team). Many of my colleagues, including myself in the Policy Team, have often wondered about the differences in scope between the two teams and whether the DS team will eventually replace us. Some even believe that the term “Data Science” sounds much fancier than “Policy” and that the work of the Policy team is merely mundane.

Let me provide a brief description of the typical tasks performed by each team. The DS team focuses on building supervised learning models, such as the Application Score (A-score), Behaviour Score (B-score), and Collection Score (C-score). Essentially, their role involves creating models to predict the likelihood of default and recovery. On the other hand, the Policy team utilizes the models developed by the DS team to design underwriting policies and account management policies. This includes making decisions on approvals, determining credit limits, pricing, and tenure. It is true that the Policy team also carries out numerous mundane tasks, such as setting up the rule engine and uploading adjustments of limits, pricing, and tenure to the system.

To address the common questions regarding the scope of the two teams and the potential for one to replace the other, we need to compare the fundamental differences between Data Science and Policy (In my current company). One key distinction lies in the nature of the problems they address: classification versus sequential decision making. A classification problem involves labeling and sorting, whereas a sequential decision making problem is more akin to chess or game problems. In a game of chess, each move you make not only affects the immediate position of the pieces but also impacts the possibilities for future moves and ultimately determines whether you win or lose. In credit risk management, actions such as adjusting a user’s credit limit have consequences not only for immediate profitability but also for long-term default probabilities.

Sequential decision making problems cannot be effectively solved using classification techniques because classification models do not consider the sequence of events. While a classification model may predict whether a user will default in six months, it does not tell us when. Moreover, classification models do not account for delayed rewards. For instance, early defaults can be advantageous as they expose risks sooner, allowing accounts to be terminated and the remaining users to recover the loss later on. Classification models learn solely from training data and fail to account for the benefits of exploration. In the Policy team, it is common to test more aggressive strategies even when the model’s predictions do not suggest it. Additionally, classification models often assume a stationary environment or underlying data distributions, which rarely holds true in reality.

So, what is a policy? A policy is a set of rules designed to determine the best course of action that maximizes immediate and future benefits based on available information. By understanding the distinction between classification problems and sequential decision making problems, we can grasp the true nature of the work carried out by the Policy team. First, we are essentially playing chess and need to think a few steps ahead. Second, there is a benefit to exploration, as we encounter unseen populations. Lastly, we learn from responses and feedback.

Returning to the questions of the DS team versus the Policy team and machine versus human, the answer lies not in the teams themselves but in the fundamental difference between classification and decision making. However, if the DS team and machines can effectively solve sequential decision making problems with incomplete information, they may indeed replace the Policy team and humans. Machines have proven to be more effective in handling sequential decision making problems with complete information.