Skip to content

About

Explainable malware detection dashboard — upload a Windows PE file, compare Random Forest vs XGBoost predictions side by side, inspect SHAP/LIME explanations and run adversarial what-if analysis. Interactive companion to a published research paper. Python · FastAPI · React · Docker.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

MalVis — Malware Feature Visualization Dashboard

Interactive web dashboard for explainable malware detection. Upload a Windows PE file, get predictions from multiple ML models, and inspect why each model made its call via SHAP and LIME.

CI License: MIT Python 3.11+ FastAPI Next.js

Live demo: coming soon · Paper: link to your published paper


Why this project exists

This dashboard is the deployable, interactive companion to a published research paper on malware detection using Random Forest and XGBoost with SHAP/LIME explainability. Static papers and notebooks make it hard for reviewers, collaborators, and security teams to reason about model behavior on real samples — MalVis turns the same models into a tool you can actually use.

Three things it does that a notebook can't:

  1. Live multi-model comparison — RF vs XGBoost (vs ANN) on the same input, side by side.
  2. Per-prediction explanations — SHAP waterfall + feature importance, generated on demand.
  3. Adversarial what-if — perturb individual features and watch the prediction shift in real time.

Who this helps and the impact it creates

Malware detection has historically been a black box: a verdict appears, an analyst trusts or distrusts it, and the reasoning is invisible. MalVis exists because a verdict without a reason is not a decision — it's a guess, and in security, guesses cost time, money, and breaches.

Who benefits

  • SOC analysts and incident responders — instead of escalating an opaque "malicious" flag, they can point at the exact PE features (suspicious imports, abnormal section entropy, packer signatures) that drove the call, cutting triage time and false-positive fatigue.
  • Security researchers and academics — published papers usually freeze at static metrics. MalVis lets peers run the same trained models on their own samples, reproduce the explanations, and challenge the findings on real binaries — turning a paper into a living artifact.
  • Students and educators — a hands-on, visual entry point into explainable AI for cybersecurity. Drop a binary, see SHAP attribute the decision feature-by-feature, and learn how RF and XGBoost actually reason about malware.
  • Blue-team engineers building detection pipelines — the REST API (/scan, /explain, /perturb) is designed to slot into SIEM/SOAR workflows so explanations travel with alerts, not just labels.
  • AI safety and adversarial-ML researchers — the perturbation simulator surfaces model fragility: which single feature flip turns a "malicious" verdict benign? That's the threat model attackers already exploit, made inspectable.

The positive impact

  • Trust through transparency. Explainability is a regulatory direction of travel (EU AI Act, NIST AI RMF). Tools that show their work are how ML earns the right to be deployed in high-stakes domains like security.
  • Faster, better-informed response. Every minute saved on triage is a minute earlier that a real threat is contained. Per-feature attributions let analysts skip straight to evidence.
  • Reproducible research. Anyone can verify the paper's claims on arbitrary samples — closing the gap between "published" and "useful in practice."
  • Safer ML deployment. By making adversarial fragility visible up front, defenders learn the limits of their models before attackers do.
  • Open and accessible. MIT-licensed, runnable with one docker compose up, and free to use — lowering the barrier for under-resourced teams, students, and researchers in regions where commercial XAI tooling is out of reach.

In short: MalVis is a small but concrete step toward AI that security teams can actually understand, audit, and trust — and that's a net good for the people defending systems and the users they protect.


Features

  • Drag-and-drop PE file upload with size and format validation
  • Multi-model inference — Random Forest, XGBoost, optional ANN — with confidence scores
  • SHAP waterfall plots for any single prediction
  • Global + local feature importance views with toggle
  • Adversarial perturbation simulator — flip imports, change section entropy, see prediction delta live
  • REST API designed for SIEM-style integration (/scan, /explain, /perturb)
  • Containerized — single docker compose up to run locally

Screenshots

The animated GIF and screenshots will be added once the dashboard ships to a live URL (Phase 4). The local stack runs end-to-end today via docker compose up.

docs/screenshots/  ← reserved for the dashboard hero GIF, verdict
                     table, SHAP waterfall, and perturbation simulator.

Architecture

┌─────────────────┐         ┌─────────────────────┐         ┌──────────────────┐
│  Next.js UI     │  HTTPS  │  FastAPI backend    │   in-   │  Trained models  │
│  (Tailwind +    │ ──────► │  + SHAP/LIME runner │ process │  RF / XGBoost /  │
│   D3 viz)       │         │  + PE feature extr. │         │  ANN (joblib)    │
└─────────────────┘         └─────────────────────┘         └──────────────────┘

The feature schema is the load-bearing contract between the extractor and the trained models — see docs/ADR-001-feature-set.md for the rationale and the schema-version-as-contract pattern.

  • Frontend — Next.js (App Router) + TailwindCSS + D3.js for visualizations
  • Backend — FastAPI + scikit-learn + XGBoost + SHAP, with pefile for PE parsing
  • Models — pickled artifacts loaded once at startup (singleton registry)
  • Deployment — Docker images; backend on Railway/Fly.io, frontend on Vercel

Tech stack

Backend Python 3.11 · FastAPI · scikit-learn · XGBoost · SHAP · LIME · pefile · pydantic · joblib
Frontend Next.js 14 · TypeScript · TailwindCSS · D3.js · React Query
Infra Docker · docker-compose · GitHub Actions CI · Vercel · Railway


Getting started

Prerequisites

  • Python 3.11+
  • Node.js 20+
  • Docker (optional, for containerized run)

Local development

# Clone
git clone https://github.com/<your-username>/malvis.git
cd malvis

# Backend
cd backend
python -m venv .venv
source .venv/bin/activate            # Windows: .venv\Scripts\activate
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8000

# Frontend (in a second terminal)
cd frontend
npm install
npm run dev                           # http://localhost:3000

Visit http://localhost:3000, drop a PE file, see predictions.

Training real artifacts

The dashboard ships in mock mode by default — predictions come from deterministic placeholders so the demo runs end-to-end without any trained models. To generate real .joblib artifacts and a manifest the registry can SHA-verify and load:

cd backend && pip install -r requirements.txt && cd ..
python -m training.train
# .env at repo root:
echo "USE_MOCK_MODELS=false" >> .env
# restart the backend; mock_mode=false on /models from now on

The default script trains on a synthetic dataset (PEs built from app/utils/pe_builder.py) — enough to exercise the whole pipeline but not real malware detection. To train on EMBER, BODMAS, or a custom dataset, replace _build_synthetic_dataset per training/README.md.

Docker (one-shot)

cp .env.example .env
docker compose up --build
# open http://localhost:8080

Brings up three containers: the FastAPI backend, the Next.js frontend, and a Caddy reverse proxy on :8080. The proxy serves the frontend at / and routes /api/* to the backend (with the /api prefix stripped). The backend runs the PE feature extractor as a subprocess with RLIMIT_AS/RLIMIT_CPU/timeout for OS-level isolation; the SECURITY.md container-level isolation (per-request sidecar) is deferred to the production deploy step.


API

Method Endpoint Description
POST /scan Upload PE file → predictions from all loaded models.
GET /scan/{sha256} Permalink lookup — cached predictions, plus sample fallback.
POST /explain SHAP values for an upload, ready for the waterfall renderer.
GET /explain/{sha256} SHAP values for a gallery sample (uploads aren't retained).
POST /perturb Apply a perturbation spec to a feature vector → before/after deltas.
GET /samples List the curated synthetic-PE gallery.
POST /samples/{id}/scan Run a gallery sample through the full scan pipeline.
GET /models Loaded model metadata + feature schema contract.
GET /ready Readiness probe — gates on the model registry being loaded.
GET /health Liveness probe — process is up.

OpenAPI / Swagger

When the backend is running, the auto-generated Swagger UI lives at /docs (or /api/docs behind the Caddy proxy). For offline use, the spec is committed at docs/openapi.json. To regenerate after an endpoint change:

cd backend && pip install -r requirements.txt && cd ..
python -m scripts.export_openapi

CI will eventually gate on docs/openapi.json matching what app.openapi() produces — drift between the two is a sign someone changed an endpoint without updating the contract.

Postman

Postman imports OpenAPI 3.x natively — no separate collection file needed:

  1. Postman → Import → File → select docs/openapi.json.
  2. Postman creates a collection with one folder per tag and one request per operation.

curl quickstart

The examples below assume the backend is on http://localhost:8000 (the default for uvicorn app.main:app --reload). Behind the Caddy proxy in docker compose, prefix every path with /api/ (http://localhost:8080/api/scan, etc).

# 1. Liveness + readiness
curl -s http://localhost:8000/health
curl -s http://localhost:8000/ready

# 2. List loaded models and the feature contract they were trained on
curl -s http://localhost:8000/models | jq

# 3. List the gallery samples; pick one and scan it
curl -s http://localhost:8000/samples | jq
curl -s -X POST http://localhost:8000/samples/minimal-clean/scan | jq

# 4. Permalink lookup by SHA-256 (cache hit OR sample fallback)
SHA=$(curl -s http://localhost:8000/samples | jq -r '.samples[0].sha256')
curl -s "http://localhost:8000/scan/$SHA" | jq

# 5. Upload your own PE file
curl -s -X POST http://localhost:8000/scan \
     -F "upload=@/path/to/sample.exe;type=application/octet-stream" | jq

# 6. SHAP explanation for a sample (rebuilt from the registry)
curl -s "http://localhost:8000/explain/$SHA?model_name=rf-v1&top_n=8" | jq

# 7. Perturbation simulator — flip the signature bit and ask for a delta
curl -s -X POST http://localhost:8000/perturb \
     -H "Content-Type: application/json" \
     -d '{
       "feature_vector": '"$(curl -s http://localhost:8000/samples/minimal-clean/scan | jq '[.features | (
         [.size_bytes, .machine, .number_of_sections, .time_date_stamp, .characteristics,
          .magic, .major_linker_version, .minor_linker_version, .size_of_code,
          .size_of_initialized_data, .size_of_uninitialized_data, .address_of_entry_point,
          .image_base, .section_alignment, .file_alignment, .subsystem,
          .dll_characteristics, .size_of_image, .size_of_headers,
          .imported_dll_count, .imported_function_count,
          (if .has_signature then 1 else 0 end), (if .has_resources then 1 else 0 end),
          (if .has_debug then 1 else 0 end), (if .has_tls then 1 else 0 end),
          0, 0, 0])
       ]')"',
       "perturbations": [{"feature": "has_signature", "op": "set", "value": 1}]
     }' | jq

The perturbation example builds the feature vector inline from /samples/.../scan. In a real client you'd cache the vector after the first scan and feed it back as-is.


Project structure

malvis/
├── backend/
│   ├── app/
│   │   ├── main.py
│   │   ├── routers/         scan, explain, perturb, models
│   │   ├── services/        pe_features, model_registry, shap_runner
│   │   └── schemas/         pydantic models
│   ├── tests/
│   ├── Dockerfile
│   └── requirements.txt
├── frontend/                Next.js app
├── notebooks/               training + experimentation
├── artifacts/               trained model pickles (gitignored)
├── docs/                    architecture decision records
├── docker-compose.yml
└── README.md

Roadmap

  • Phase 1 — Backend skeleton + model loading
  • Phase 2 — /scan, /explain, /perturb, /models, /ready endpoints + sandboxed extractor
  • Phase 3 — Next.js dashboard (upload, verdict table, SHAP waterfall, perturbation simulator wired)
  • [~] Phase 4 — Dockerize + deploy live URL (compose stack runs locally; hosting decision + live URL pending)
  • Phase 5 — Demo video + documentation polish
  • Phase 6 — REST API auth, batch scan, model versioning

See TODO.md for the full breakdown.


Research context

MalVis productionizes the models and findings from a peer-reviewed paper on explainable malware detection. The dashboard lets reviewers and security practitioners interact with the same Random Forest and XGBoost classifiers described in the paper, on arbitrary PE samples, with the same SHAP-based interpretability layer.

Citation will be added here once the live demo links back from the paper.


Contributing

This is primarily a portfolio project, but issues and PRs are welcome. See CONTRIBUTING.md for development setup, code-style conventions, and the security-disclosure path.


License

MIT — see LICENSE.


Author

Md. Rakibul Hasan — devrakib.com
Backend developer at Combosoft · Master's applicant in Data Science (Germany, intake 2026/27)

About

Explainable malware detection dashboard — upload a Windows PE file, compare Random Forest vs XGBoost predictions side by side, inspect SHAP/LIME explanations and run adversarial what-if analysis. Interactive companion to a published research paper. Python · FastAPI · React · Docker.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages