Skip to content

Commit 144ebf1

Browse files
webknjazwebknjaz
authored
Merge pull request #1366 from EpicWink/more-indexes-2
Add more package index projects
2 parents c8f51ab + 35c7f84 commit 144ebf1

3 files changed

Lines changed: 305 additions & 33 deletions

File tree

‎source/guides/hosting-your-own-index.rst‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,10 +52,86 @@ directory with autoindex enabled. For an example using the built in Web server
5252
in `Twisted`_, you would simply run ``twistd -n web --path .`` and then
5353
instruct users to add the URL to their installer's configuration.
5454

55+
56+
Existing projects
57+
=================
58+
59+
.. list-table::
60+
:header-rows: 1
61+
62+
* - Project
63+
- Package upload
64+
- PyPI fall-through [2]_
65+
- Additional notes
66+
67+
* - :ref:`devpi`
68+
- ✔
69+
- ✔
70+
- multiple indexes with inheritance, with syncing, replication, fail-over;
71+
mirroring
72+
73+
* - :ref:`simpleindex`
74+
-
75+
- ✔
76+
-
77+
78+
* - :ref:`pypiserver`
79+
- ✔
80+
-
81+
-
82+
83+
* - :ref:`pypiprivate`
84+
-
85+
-
86+
-
87+
88+
* - :ref:`pypicloud`
89+
-
90+
-
91+
- unmaintained; also cached proxying; authentication, authorisation
92+
93+
* - :ref:`pywharf`
94+
-
95+
-
96+
- unmaintained; serve files in GitHub
97+
98+
* - :ref:`pulppython`
99+
- ✔
100+
-
101+
- also mirroring, proxying; plugin for Pulp
102+
103+
* - :ref:`pip2pi`
104+
-
105+
-
106+
- also mirroring; manual synchronisation
107+
108+
* - :ref:`dumb-pypi`
109+
-
110+
-
111+
- not a server, but a static file site generator
112+
113+
* - :ref:`httpserver`
114+
-
115+
-
116+
- standard-library
117+
118+
* - `Apache <https://httpd.apache.org/>`_
119+
-
120+
- ✔
121+
- using
122+
`mod_rewrite
123+
<https://httpd.apache.org/docs/current/mod/mod_rewrite.html>`_
124+
and
125+
`mod_cache_disk
126+
<https://httpd.apache.org/docs/current/mod/mod_cache_disk.html>`_,
127+
you can cache requests to package indexes through an Apache server
128+
55129
----
56130

57131
.. [1] For complete documentation of the simple repository protocol, see
58132
:ref:`simple repository API <simple-repository-api>`.
59133
134+
.. [2] Can be configured to fall back to PyPI (or another package index)
135+
if a requested package is missing.
60136
61137
.. _Twisted: https://twistedmatrix.com/

‎source/guides/index-mirrors-and-caches.rst‎

Lines changed: 86 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -5,19 +5,24 @@ Package index mirrors and caches
55
================================
66

77
:Page Status: Incomplete
8-
:Last Reviewed: 2014-12-24
8+
:Last Reviewed: 2023-11-08
99

10-
11-
Mirroring or caching of PyPI can be used to speed up local package installation,
10+
Mirroring or caching of PyPI (and other
11+
:term:`package indexes <Package Index>`) can be used to speed up local
12+
package installation,
1213
allow offline work, handle corporate firewalls or just plain Internet flakiness.
1314

14-
Three options are available in this area:
15+
There are multiple classes of options in this area:
16+
17+
1. local/hosted caching of package indexes.
18+
19+
2. local/hosted mirroring of a package index. A mirror is a (whole or
20+
partial) copy of a package index, which can be used in place of the
21+
original index.
1522

16-
1. pip provides local caching options,
17-
2. devpi provides higher-level caching option, potentially shared amongst
18-
many users or machines, and
19-
3. bandersnatch provides a local complete mirror of all PyPI :term:`packages
20-
<Distribution Package>`.
23+
3. private package index with fall-through to public package indexes (for
24+
example, to mitigate dependency confusion attacks), also known as a
25+
proxy.
2126

2227

2328
Caching with pip
@@ -38,26 +43,75 @@ cached copies of :term:`packages <Distribution Package>`:
3843
python3 -m pip install --no-index --find-links=/tmp/wheelhouse SomeProject
3944
4045
41-
Caching with devpi
42-
------------------
43-
44-
devpi is a caching proxy server which you run on your laptop, or some other
45-
machine you know will always be available to you. See the `devpi
46-
documentation for getting started`__.
47-
48-
__ https://devpi.net/docs/devpi/devpi/latest/+d/quickstart-pypimirror.html
49-
50-
51-
Complete mirror with bandersnatch
52-
----------------------------------
53-
54-
bandersnatch will set up a complete local mirror of all PyPI :term:`packages
55-
<Distribution Package>` (externally-hosted packages are not mirrored). See
56-
the `bandersnatch documentation for getting that going`__.
57-
58-
__ https://github.com/pypa/bandersnatch/
59-
60-
A benefit of devpi is that it will create a mirror which includes
61-
:term:`packages <Distribution Package>` that are external to PyPI, unlike
62-
bandersnatch which will only cache :term:`packages <Distribution Package>`
63-
hosted on PyPI.
46+
Existing projects
47+
-----------------
48+
49+
.. list-table::
50+
:header-rows: 1
51+
52+
* - Project
53+
- Cache
54+
- Mirror
55+
- Proxy
56+
- Additional notes
57+
58+
* - :ref:`devpi`
59+
- ✔
60+
- ✔
61+
-
62+
- multiple indexes with inheritance; syncing, replication, fail-over;
63+
package upload
64+
65+
* - :ref:`bandersnatch`
66+
- ✔
67+
- ✔
68+
-
69+
-
70+
71+
* - :ref:`simpleindex`
72+
-
73+
-
74+
- ✔
75+
- custom plugin enables caching; re-routing to other package indexes
76+
77+
* - :ref:`pypicloud`
78+
- ✔
79+
-
80+
- ✔
81+
- unmaintained; authentication, authorisation
82+
83+
* - :ref:`pulppython`
84+
-
85+
- ✔
86+
- ✔
87+
- plugin for Pulp; multiple proxied indexes; package upload
88+
89+
* - :ref:`proxpi`
90+
- ✔
91+
-
92+
- ✔
93+
- multiple proxied indexes
94+
95+
* - :ref:`nginx_pypi_cache`
96+
- ✔
97+
-
98+
- ✔
99+
- multiple proxied indexes
100+
101+
* - :ref:`flaskpypiproxy`
102+
- ✔
103+
-
104+
- ✔
105+
- unmaintained
106+
107+
* - `Apache <https://httpd.apache.org/>`_
108+
- ✔
109+
-
110+
- ✔
111+
- using
112+
`mod_rewrite
113+
<https://httpd.apache.org/docs/current/mod/mod_rewrite.html>`_
114+
and
115+
`mod_cache_disk
116+
<https://httpd.apache.org/docs/current/mod/mod_cache_disk.html>`_,
117+
you can cache requests to package indexes through an Apache server

0 commit comments

Comments
 (0)