Finance · Advanced
Predict which Kenyan digital-loan customers will default, from 68,000 loan records and FRED macro indicators, and ship the model as an API.
Open in SQL workspace
Open in Python Lab
How to do this projectdata/ folder of the Python Lab.You are a data scientist at a Kenyan digital lender that funds short-term loans together with partner lenders. The credit team wants a machine-learning system that predicts whether a customer will default, so high-risk borrowers can be priced or declined before money goes out. You have historical loan, customer and lender records: some customers appear many times, and a single loan can appear with several lenders because loans are co-funded. You also have macroeconomic indicators from the FRED portal to enrich the picture. Your job is to explore the data, engineer features, train and compare classifiers, save the best pipeline, and deploy it behind Flask or FastAPI.
Your role
Data Scientist, credit risk team at a Kenyan digital lender
Financial institutions lose money when customers fail to repay, and traditional credit scoring does not fully capture borrower behaviour or the economic conditions loans are issued into. The lender needs a model that accurately predicts default from historical loan, customer and economic data, and a way to call it on new applications.
Create a virtual environment, install the suggested libraries, and lay out the project folders (data/raw, data/processed, notebooks, models, app, src). Load both CSV files, inspect shape and data types, explore missing values and understand the target distribution.
Handle missing values, duplicate records, incorrect data types, date formatting and outliers. Convert disbursement and due dates to datetime, check for duplicate rows, and decide how to treat the same loan appearing with several lenders.
Univariate analysis of loan amounts, durations and the target; bivariate analysis of loan type, new versus repeat and amount against the target; multivariate views with correlation matrices and grouped statistics, using histograms, box plots, count plots, heatmaps and scatter plots.
Investigate numerical features against the target with a correlation matrix, point-biserial correlation and feature importance; categorical features with chi-square tests, crosstabs and grouped target distributions. Check multicollinearity and identify redundant features.
Create date-based features (loan month, year, duration category), financial features (interest_amount = Total_Amount_to_Repay − Total_Amount, repayment_ratio = Total_Amount_to_Repay / Total_Amount) and customer-based features (loan frequency, previous borrowing patterns, aggregated customer statistics). Join the FRED indicators to each loan by country and disbursement year.
Encode categoricals, scale numerics and build the preprocessing with ColumnTransformer inside a scikit-learn Pipeline. Train logistic regression, decision tree, random forest and at least one of XGBoost, LightGBM or CatBoost, with cross-validation on a stratified split.
Compare models on the held-out set with accuracy, precision, recall, F1 and ROC-AUC, plot confusion matrices and ROC curves, choose the final model with a justification, and save both the model and the preprocessing pipeline with joblib.
Build a Flask or FastAPI app that receives user input, applies the preprocessing pipeline, generates a prediction and returns the result. Write the README, freeze requirements and push the repository.
Two files. The loans table has 68,654 rows and 16 columns: row ID, customer, country, loan and lender identifiers, loan type, amount disbursed and amount to repay, disbursement and due dates, duration in days, whether the customer is new or repeat, the lender's funded amount, share and expected repayment, and the binary target (1 = defaulted, 0 = repaid). The economic indicators file is a wide table of 27 country-indicator rows with one column per year from 2001 to 2023, covering inflation, exchange rate, real, deposit and lending interest rates, interest rate spread, precipitation, fossil fuel energy use and unemployment for several African countries.
The scenario
Objectives
Tasks and business questions
Dataset and data dictionarycredit_loans68,654 loan-lender rows with customer, loan, lender and repayment details and the default target. · 68,654 rows
| Column | Type | Description | Example |
|---|---|---|---|
ID | text | Unique identifier for each row. | ID_257244237466267278 |
customer_id | integer | Unique customer identifier; a customer can appear on many rows. | 257244 |
country_id | text | Country where the loan was issued. | Kenya |
tbl_loan_id | integer | Unique loan identifier; a loan appears once per funding lender. | 237466 |
lender_id | integer | Identifier of the lender funding this portion. | 267278 |
loan_type | text | Category of loan. | Type_1 |
Total_Amount | decimal |
economic_indicators27 country-indicator rows, one column per year 2001 to 2023: inflation, exchange rate, interest rates, precipitation, energy use and unemployment. · 27 rows
| Column | Type | Description | Example |
|---|---|---|---|
Country | text | Country the indicator describes. | Kenya |
Indicator | text | Indicator name, e.g. inflation (FP.CPI.TOTL.ZG), exchange rate (PA.NUS.FCRF), real, deposit and lending interest rates, interest rate spread, precipitation, fossil fuel energy use, unemployment. |
11 exercises worth 185 points. Your query runs against this project’s dataset and is marked automatically.
8 exercises worth 120 points. Your code runs in the Python Lab against this project’s data and is marked automatically.
Marked with the machine-learning rubric. Expect the target imbalance to be measured and to shape the metric choice; repeated customers and co-funded loans to be handled without leaking the target across rows of the same loan; features such as interest amount, repayment ratio and customer loan frequency to be justified; several model families compared on a held-out set with ROC-AUC and recall on the default class reported; and a deployed endpoint that returns a prediction for a new loan.
| Total loan amount disbursed to the customer. |
16477.0 |
Total_Amount_to_Repay | decimal | Total the customer must repay, including interest and fees. | 16642.0 |
disbursement_date | date | Date the loan was issued. | 2022-08-15 |
due_date | date | Date repayment is due. | 2022-08-22 |
duration | integer | Loan term in days. | 7 |
New_versus_Repeat | text | Whether this is the customer's first loan or a repeat loan. | Repeat Loan |
Amount_Funded_By_Lender | decimal | Portion of the loan funded by this lender. | 2436.0 |
Lender_portion_Funded | decimal | Share of the total loan funded by this lender (0 to 1). | 0.1478 |
Lender_portion_to_be_repaid | decimal | Amount this lender expects to recover. | 2460.0 |
target | integer | 1 = customer defaulted, 0 = customer repaid. | 0 |
Inflation, consumer prices (annual %) |
YR2001 … YR2023 | decimal | One column per year with the indicator's value; unpivot these to join by year. | 7.99 |
Clean, labelled notebooks following the expected structure: data loading, cleaning, EDA, feature engineering, model training and model evaluation