Class Imbalance in AI Compliance Models: Why Rare Cases Create Asymmetric Legal Risk

Summary: Class imbalance in artificial intelligence-based compliance models creates a distinctive legal and governance risk because the transactions least represented in historical training data are often those carrying the greatest regulatory consequences. While machine learning models can classify routine transactions with high accuracy, they may perform poorly on unusual transactions involving overlapping statutory provisions, uncertain vendor status, recent legislative amendments or disputed tax interpretations. Consequently, high overall model accuracy can conceal serious weaknesses in legally sensitive classifications. The article illustrates this problem through the distinction between Sections 194C and 194J of the Income-tax Act, 1961. An automated system may confidently classify a mixed software development and deployment contract based on familiar training patterns without adequately recognising the legal characteristics governing the applicable TDS provision. Incorrect classification can expose the deductor to short-deduction demands, interest and penalties. Conventional techniques such as SMOTE, class weighting and decision-threshold adjustments may improve performance but cannot independently establish legal correctness or compensate for missing regulatory knowledge. Effective compliance governance therefore requires calibrated confidence assessments, risk-based human review, transparent data lineage and separate monitoring of classification errors in legally consequential categories. The article argues that responsibility for determining acceptable regulatory risk must rest with the compliance function rather than exclusively with data scientists or system engineers. Evaluation should focus not merely on overall accuracy, but on where errors occur, their legal consequences and whether the automated system can explain the evidentiary basis for its decisions.

Introduction

Every compliance model is trained on historical data. That data is almost never balanced.

Standard cases dominate. Edge cases are rare by definition. The model sees ten thousand routine vendor payments for every fifty payments that sit in a legally ambiguous category. It learns the routine well. It learns the edge cases poorly — or not at all.

This is called class imbalance. In most machine learning contexts, it is a performance problem. You fix it with resampling techniques, class weights, or threshold tuning. You rerun your metrics. You move on.

In a compliance context, class imbalance is not a performance problem. It is an asymmetric legal risk. And the asymmetry runs exactly in the wrong direction.

What Asymmetric Legal Risk Means

A model trained on imbalanced data makes errors unevenly. It is highly accurate on majority classes — the common, well-represented cases. It is unreliable on minority classes — the rare, underrepresented ones.

Here is the problem: in compliance, the minority classes are almost always the high-stakes cases.

Common vendor payments under clear, well-established TDS sections — these are the majority. The model handles them well. Getting these right saves time. Getting them wrong is inconvenient.

Payments at the boundary of multiple sections — a contract that could be classified under 194C or 194J depending on interpretation. Payments to vendors whose residency status is unclear. Transactions involving newly notified provisions with limited historical data. These are the minority. The model handles them poorly.

And these are exactly the transactions that generate demand notices, penalty proceedings, and interest liabilities.

The error rate is highest where the regulatory consequence of error is also highest. That is the asymmetry.

A Concrete Example: The Boundary Classification Problem

Consider a model trained on three years of vendor payment data to classify TDS sections.

Section 194C — works contracts — accounts for 40% of transactions in the training data. The model has seen thousands of examples. It classifies 194C reliably.

Section 194J — professional services — accounts for 25% of transactions. Also well-represented. Also reliable.

Payments at the 194C/194J boundary — contracts that have both a service element and a works element, where classification depends on which is primary — account for 2% of historical transactions. The model has seen very few. It learns a rough approximation that works most of the time, fails on anything unusual.

Now a vendor submits a contract for developing and deploying a software system on the client’s infrastructure. Is this primarily a works contract (194C at 2%) or a professional/technical service (194J at 10%)?

The model has seen limited examples of this exact pattern. It defaults to the pattern it knows best. It classifies with high confidence. It may be right. It may be wrong.

If wrong — the deduction was at 2% on a transaction that should have been 10%. The shortfall, plus interest at 1% per month from the date of deduction, plus potential penalty of 100% to 300% of the tax shortfall, creates a liability that is an order of magnitude larger than the original classification error.

The model’s confidence score says 87%. The audit trail records a confident, automated decision. The liability exists regardless.

Why Standard Imbalance Fixes Are Insufficient for Compliance

Data scientists have standard tools for class imbalance. SMOTE generates synthetic minority samples. Class weights penalise misclassification of minority classes more heavily during training. Threshold tuning adjusts the decision boundary to reduce minority class false negatives.

These are legitimate techniques. In compliance, they are necessary but not sufficient. Here is why.

They improve average performance across classes

They do not eliminate minority class unreliability. A model trained with SMOTE on boundary classification cases will perform better on those cases than one trained without it. It will still perform worse on boundary cases than on clear-cut cases. The asymmetry shrinks; it does not disappear.

They do not change what the model does not know

If a new provision was introduced six months ago and there are fifty transactions in the training data classified under it, the model has fifty examples to learn from. SMOTE cannot manufacture understanding. It can generate synthetic samples from those fifty, but the synthetic samples inherit the limitations of the fifty originals.

They do not produce human-readable uncertainty

A compliance model needs to know not just what it decided, but how confident it should be in that decision — and specifically whether that confidence is grounded in robust training data or in extrapolation from limited examples.

What Compliance-Aware Handling of Imbalanced Data Requires

There are three requirements that go beyond standard ML practice.

1. Confidence must be calibrated, not just computed

Raw model confidence scores are not calibrated to reflect training data density. A model that has seen ten thousand examples of a class and one that has seen fifty examples can both output confidence scores of 87%. The scores are not comparable.

Calibrated confidence — confidence that reflects how much training data the model is actually basing its decision on — is what compliance systems need. A decision made with high confidence from abundant training data is different from a decision made with high confidence from sparse training data. The audit trail must distinguish between them.

2. Low-confidence decisions must route to human review, not to the output queue

This is not optional. Any system making legally consequential classifications must have a confidence threshold below which decisions are not automated. The threshold is not a technical parameter — it is a governance decision made by the compliance function, not the data science team.

What that threshold should be depends on the regulatory consequence of misclassification in each category. For provisions where misclassification creates penalty exposure, the threshold should be high. For routine, well-established classifications, it can be lower.

3. The audit trail must record data lineage alongside the decision

When a compliance decision is logged, the log should capture not just the decision and the rule applied — but the confidence level and, where available, the training data density for the relevant class. An auditor reviewing a high-risk classification should be able to see that the model made this decision with 82% confidence based on 47 training examples in a category with significant recent regulatory activity.

That is a materially different record than one that simply shows “classified: Section 194J, confidence 82%.”

The Governance Question

Who is responsible for the class imbalance problem in a compliance model?

Not the data scientist who trained it. They optimised for the metrics they were given.

Not the engineer who deployed it. They deployed what they were handed.

The compliance function owns this problem. Because the compliance function is the one that understands which misclassifications create regulatory exposure and which do not. The compliance function is the one that can define appropriate confidence thresholds by risk category. And the compliance function is the one that will answer to a regulator when the model’s errors are concentrated exactly where the regulatory stakes are highest.

A compliance model that has not had its class imbalance reviewed by the compliance function — not the data science team — has not been adequately governed.

The question to ask of any deployed compliance model is not “what is the overall accuracy?” The question is: “where are the errors concentrated, and what is the regulatory consequence of errors in those specific categories?”

If no one has asked that question, the answer is probably the one you do not want.