Testing Policy Thresholds
On Mandi Margins
MandiIQ is an open-source analytical warehouse and dashboard that applies Causal Regression Discontinuity Designs (RDD) to test whether IMD rainfall-deficit thresholds drive structural margins in national commodity markets.
Warehouse Scale & Coverage
Regression Discontinuity Design
Does the IMD declaring a region as "rain-deficient" drive agricultural market behavior, or is price movement merely a continuous function of physical precipitation?
We test this by placing rain departure on the horizontal axis and modeling the discontinuity at the official -20% deficit cutoff.
Causal RDD Specification
We estimate local linear regressions on both sides of the -20% threshold:
Yit = α + β Dit + γ1 (Xit - c) + γ2 Dit(Xit - c) + εit
Where Yit is log market price, Xit is rainfall departure %, c = -20% is the threshold, and Dit is indicator for deficit territory.
Our findings suggest that while physical rainfall levels strongly affect yields, the administrative -20% declaration produces statistically insignificant jumps in most crops, with the notable exception of Onions which show active hoarding signals.
Decoupled Processing & Serving
MandiIQ decouples the analytical warehouse from the customer-facing dashboard. The system runs an automated ELT pipeline using DuckDB, fits locally weighted regressions, tracks models in MLflow, and exposes predictions through FastAPI.
%%{init: {"theme": "dark", "themeVariables": { "primaryColor": "#1a1a2e", "primaryTextColor": "#fff", "lineColor": "#d7ff00"}}}%%
flowchart TB
subgraph Sources["Data Sources"]
A1["Agmarknet API
Daily Mandi Prices"]
A2["IMD Weather Grids
Rainfall Departure"]
A6["Ashoka CEDA
Historical Price CSV"]
end
subgraph Ingestion["Ingestion Pipeline"]
B1["fetch_prices.py
Pagination + Retry"]
B2["fetch_rainfall.py
Sub-division Parser"]
B3["fetch_ndvi.py
Sentinel Hub"]
B4["scheduler.py
Orchestration"]
end
subgraph Storage["Analytical Store"]
C1["DuckDB
mandi_iq.duckdb
{{n_prices}} rows"]
C2["data_lineage
Provenance Tracking"]
end
subgraph Analysis["Analysis Engine"]
D1["Causal RDD
Local-linear + FE"]
D2["Forecast
Seasonal Naive"]
D3["Classifier
XGBoost + SHAP"]
end
subgraph Serving["Serving Layer"]
F1["FastAPI Gateway
/health /prices /forecast"]
F2["Streamlit Dashboard
test-mandi-keae7eruks2n4cqvumjfu8.streamlit.app"]
end
A1 --> B1; A6 --> B3; A2 --> B2; B4 --> B1; B4 --> B2; B4 --> B3; B1 --> C1; B2 --> C1; B3 --> C1; C1 --> D1; C1 --> D2; C1 --> D3; D1 --> F1; D2 --> F1; D3 --> F1; F1 --> F2;
Failure Modes
If the MLflow registry is unreachable, FastAPI falls back to a local serialized model pickle. Climate variables exceeding 3σ trigger automated retraining on a 90-day window. All forecasting endpoints gracefully fallback to moving-average baselines.
All endpoints fallback gracefully. If an API call fails, the dashboard swaps to a historical moving-average baseline.
Repository Map 87 Python · 10 Modules · 14 Dashboard Pages · 5 SQL
Click any file to open it on GitHub.
- orchestrator.py
- router.py
- models.yaml
- rdd_engine.py Causal RDD
- fixed_effects.py FE cross-check
- forecast.py
- classifier.py
- prescriptive.py
- fetch_prices.py 200ms
- fetch_rainfall.py 1m 24s
- fetch_ndvi.py 3m 0s
- scheduler.py
- executive_overview
- discontinuity
- forecast
- risk_map
- satellite
- causal_explorer
- api/main.py
- storage/duckdb_store.py
- styles/design.css
- tests/test_verification.py
- FlipBoard.tsx
- WebGLHero.tsx
- vite.config.ts
Live Pipeline Status loading...
| Commodity | Latest Date | Earliest Date | Rows | Districts | States |
|---|---|---|---|---|---|
| Loading... | |||||