Skip to content

feat: add @pgpm/geolite — GeoLite2 IP geolocation module#76

Open
pyramation wants to merge 2 commits into
mainfrom
feat/geolite-module
Open

feat: add @pgpm/geolite — GeoLite2 IP geolocation module#76
pyramation wants to merge 2 commits into
mainfrom
feat/geolite-module

Conversation

@pyramation
Copy link
Copy Markdown
Contributor

Summary

Adds @pgpm/geolite — a new pgpm module providing GeoLite2 IP geolocation tables and lookup functions for PostgreSQL.

pgpm extension (schema DDL, deployed via pgpm deploy):

  • geolite.network — CIDR blocks → geoname locations + lat/long/accuracy (~4M rows for City IPv4), GiST indexed for >>= containment lookups
  • geolite.location — geoname_id → country/city/subdivision/timezone metadata, UNIQUE on (geoname_id, locale_code)
  • geolite.asn — CIDR blocks → autonomous system number/organization, GiST indexed
  • geolite.data_version — tracks loaded release version
  • geolite.lookup(inet) / geolite.lookup_asn(inet) — convenience functions joining network + location
  • All tables use uuidv7() primary keys and GRANT SELECT TO public for global readability

TypeScript loader (src/loader.ts):

  • Downloads .mmdb files from P3TERX/GeoLite.mmdb (no MaxMind account required)
  • Walks the MMDB binary trie by scanning IPv4/IPv6 space and skipping by prefix length
  • Streams records into PostgreSQL via COPY for fast bulk loading
  • TRUNCATE + COPY for atomic reload; re-runnable for weekly updates
  • Optional --ipv6 flag for including IPv6 blocks
SELECT * FROM geolite.lookup('8.8.8.8'::inet);
SELECT * FROM geolite.lookup_asn('8.8.8.8'::inet);

Ref: https://github.com/constructive-io/constructive-planning/issues/356

Review & Testing Checklist for Human

  • Review table schemas — verify column types match MaxMind's GeoLite2 data model
  • Verify GRANT SELECT TO public is the right access level for your use case
  • Deploy locally (pgpm deploy) and run the loader (DATABASE_URL=... pnpm geolite:load) to confirm end-to-end
  • Test lookup functions with known IPs (e.g. SELECT * FROM geolite.lookup('8.8.8.8'::inet))

Notes

  • The loader iterates the full IPv4 space (~4M CIDR blocks for City), which takes a few seconds. IPv6 is opt-in via --ipv6.
  • Data sourced from P3TERX/GeoLite.mmdb (CC BY-SA 4.0, mirrors MaxMind's GeoLite2 weekly).
  • Uses native PostgreSQL cidr + GiST — no ip4r or PostGIS dependencies. ip4r can be added later for high-throughput use cases.
  • maxmind and pg-copy-streams are devDependencies since the loader runs outside pgpm's deploy lifecycle.

Link to Devin session: https://app.devin.ai/sessions/351581b79f0945ffbf53e0912b40a374
Requested by: @pyramation

- geolite schema with tables: network, location, asn, data_version (all with uuidv7 PKs)
- GiST indexes on network and asn for fast CIDR containment lookups (>>=)
- Lookup functions: geolite.lookup(inet) and geolite.lookup_asn(inet)
- All tables globally readable (GRANT SELECT TO public)
- TypeScript loader: downloads .mmdb from P3TERX/GeoLite.mmdb, walks MMDB trie, bulk-loads via COPY
- Follows pgpm module conventions (deploy/revert/verify triplets)
@devin-ai-integration
Copy link
Copy Markdown

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@socket-security
Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Added@​types/​pg-copy-streams@​1.2.51001007180100
Addedmaxmind@​4.3.2910010010087100

View full report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant