feat: update bk-storages to 2.0.0 (TencentBlueKing#195)

Author: narasux

.github/workflows/bk-storages.yml

@@ -2,60 +2,62 @@ name: CI Check for bk-storages
 
 on:
   push:
-    branches: [ master, staging ]
+    branches: [master, staging]
     paths:
-      - 'sdks/bk-storages/**'
+      - "sdks/bk-storages/**"
   pull_request:
-    branches: [ master, staging ]
+    branches: [master, staging]
     paths:
-      - 'sdks/bk-storages/**'
+      - "sdks/bk-storages/**"
   workflow_dispatch:
-
 jobs:
   check:
     runs-on: macos-latest
     steps:
-    - uses: actions/checkout@v2
-    - name: Set up Python
-      uses: actions/setup-python@v2
-      with:
-        python-version: 3.7
-    - name: Format with isort
-      run: |
-        pip install isort==5.9.2
-        isort sdks/ --settings-path=sdks/bk-storages/pyproject.toml
-    - name: Format with black
-      run: |
-        pip install black==21.7b0 click==8.0.4
-        black sdks/ --config=sdks/bk-storages/pyproject.toml
-    - name: Lint with flake8
-      run: |
-        pip install pyproject-flake8
-        pflake8 sdks/ --config=sdks/bk-storages/pyproject.toml
-    - name: Lint with mypy
-      run: |
-        pip install mypy==0.910 types-requests==2.25.0 types-setuptools==57.0.0 types-dataclasses==0.1.5 types-redis==3.5.4 types-PyMySQL==1.0.0 types-six==0.1.7 types-toml==0.1.3
-        mypy sdks/bk-storages --config-file=sdks/bk-storages/pyproject.toml
+      - uses: actions/checkout@v2
+      - name: Set up Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.11
+      - name: Format with isort
+        working-directory: sdks/bk-storages
+        run: |
+          pip install isort==5.12.0
+          isort . --settings-path=pyproject.toml
+      - name: Format with black
+        working-directory: sdks/bk-storages
+        run: |
+          pip install black==23.7.0 click==8.1.6
+          black . --config=pyproject.toml
+      - name: Lint with flake8
+        working-directory: sdks/bk-storages
+        run: |
+          pip install flake8==4.0.1 pyproject-flake8==0.0.1a5
+          pflake8 . --config=pyproject.toml
+      - name: Lint with mypy
+        working-directory: sdks/bk-storages
+        run: |
+          pip install mypy==0.910 types-requests==2.31.0.2 types-setuptools==57.4.18 types-dataclasses==0.1.7 types-redis==3.5.18 types-PyMySQL==1.1.0.1 types-six==0.1.9 types-toml==0.1.5
+          mypy . --config-file=pyproject.toml
   test:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ['3.7', '3.8', '3.9', '3.10']
-        poetry-version: [1.3.2]
+        python-version: ["3.8", "3.9", "3.10", "3.11"]
         os: [ubuntu-latest, macos-latest]
     runs-on: ${{ matrix.os }}
     steps:
-    - uses: actions/checkout@v2
-    - name: Set up Python
-      uses: actions/setup-python@v2
-      with:
-        python-version: ${{ matrix.python-version }}
-    - name: Set up Poetry
-      uses: abatilo/actions-poetry@v2.1.0
-      with:
-          poetry-version: ${{ matrix.poetry-version }}
-    - name: Test bk-storages
-      run: |
-        cd sdks/bk-storages
-        poetry install
-        poetry run pytest
+      - uses: actions/checkout@v2
+      - name: Set up Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Set up Poetry
+        uses: abatilo/actions-poetry@v2.3.0
+        with:
+          poetry-version: 1.5.1
+      - name: Test bk-storages
+        run: |
+          cd sdks/bk-storages
+          poetry install
+          poetry run pytest

sdks/bk-storages/CHANGE.md

@@ -1,17 +1,29 @@
 ## Change logs
+
+### 2.0.0
+
+- 支持 Python 3.11，移除对 Python 3.6, 3.7 的支持
+- Django 版本要求 >= 3.2.25
+- 禁止在保存的文件名中使用绝对路径（如：/etc/passwd）或相对路径（如：../../passwd）
+
 ### 1.1.1
+
 - 扩展支持的 Python 版本（>=3.6,<3.12)
 
 ### 1.1.0
+ 
 - 调整 `BKRepoStorage` 公开仓库的 url 实现: 直接返回资源的路径, 而并非生成临时访问地址
 
 ### 1.0.8
+ 
 - 扩展支持的 Python 版本（>=3.6,<3.11), 并修复单元测试在 django>=2.2.27 时无法通过的问题
 
 ### 1.0.7
+ 
 - 修复 `BKRepoStorage` 在保存没有配置名称的 File 对象时会触发无法上传的问题
 
 ### 1.0.6
+
 - `BKRepoStorage` 增加 DownloadFailedError 和 UploadFailedError 异常
 
 ### 1.0.5

sdks/bk-storages/README.md

@@ -162,7 +162,7 @@ storage.delete('/test/temp_file.txt')
 
 ## 开发指南
 
-首先安装 [poetry](https://github.com/python-poetry/poetry) ，之后在项目目录下执行 `poetry env use python3.6` 初始化开发用虚拟环境。然后用 `poetry shell` 命令激活虚拟环境。
+首先安装 [poetry](https://github.com/python-poetry/poetry) ，之后在项目目录下执行 `poetry env use python3.11` 初始化开发用虚拟环境。然后用 `poetry shell` 命令激活虚拟环境。
 
 - 执行 `poetry install` 安装所有依赖
 - 使用 `pytest -s .` 执行所有单元测试
@@ -174,12 +174,3 @@ storage.delete('/test/temp_file.txt')
 ### 发布包
 
 首先，执行 `poetry build` 命令在 dist 目录下生成当前版本的包。然后执行 `twine upload dist/* --repository-url {pypi_address} --username {your_name} --password {your_token}` 将其上传到 pypi 服务器上。
-
-### 关于 setup.py
-
-虽然在 [PEP 517](https://python-poetry.org/docs/pyproject/#poetry-and-pep-517) 规范里，Python 包不再需要 `setup.py` 文件。但真正少了 `setup.py` 文件后，会发现有些功能就没法正常使用，比如 pip 的可编辑安装模式、tox 等（ [相关文档](https://github.com/python-poetry/poetry/issues/761) ）。所以我们仍然需要它。
-
-为了避免维护重复的 `pyproject.toml` 和 `setup.py` 文件，我们使用了 [dephell](https://github.com/dephell/dephell) 工具来自动生成 `setup.py` 文件。
-
-- 安装 dephell
-- 在根目录执行 `dephell deps convert --from pyproject.toml --to setup.py`

sdks/bk-storages/bkstorages/__init__.py

@@ -8,4 +8,4 @@
  * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
  * specific language governing permissions and limitations under the License.
 """
-__version__ = "1.1.1"
+__version__ = "2.0.0"

sdks/bk-storages/bkstorages/backends/rgw.py

@@ -23,7 +23,7 @@
 from django.core.files.base import File
 from django.core.files.storage import Storage
 from django.utils.deconstruct import deconstructible
-from django.utils.encoding import filepath_to_uri, force_bytes, force_text, smart_str
+from django.utils.encoding import filepath_to_uri, force_bytes, force_str, smart_str
 from django.utils.timezone import is_naive, localtime
 from six import BytesIO, string_types
 from six.moves.urllib import parse as urlparse
@@ -46,8 +46,8 @@ def percent_encode(input_str, safe=SAFE_CHARS, encoding=None):
     encoding = encoding or setting('RGW_FILE_NAME_CHARSET', 'utf-8')
 
     if not isinstance(input_str, string_types):
-        input_str = force_text(input_str, encoding)
-    return quote(force_text(input_str, encoding).encode('utf-8'), safe=safe)
+        input_str = force_str(input_str, encoding)
+    return quote(force_str(input_str, encoding).encode('utf-8'), safe=safe)
 
 
 # Start patch botocore to fix Non-ascii filename exception
@@ -408,7 +408,7 @@ def _encode_name(self, name):
         return smart_str(name, encoding=self.file_name_charset)
 
     def _decode_name(self, name):
-        return force_text(name, encoding=self.file_name_charset)
+        return force_str(name, encoding=self.file_name_charset)
 
     def _compress_content(self, content):
         """Gzip a given string content."""

sdks/bk-storages/bkstorages/utils.py

@@ -13,7 +13,7 @@
 
 from django.conf import settings
 from django.core.exceptions import ImproperlyConfigured, SuspiciousFileOperation
-from django.utils.encoding import force_text
+from django.utils.encoding import force_str
 from six.moves.urllib.parse import urljoin
 
 
@@ -70,9 +70,9 @@ def safe_join(base, *paths):
     Paths outside the base path indicate a possible security
     sensitive operation.
     """
-    base_path = force_text(base)
+    base_path = force_str(base)
     base_path = base_path.rstrip('/')
-    paths = [force_text(p) for p in paths]
+    paths = [force_str(p) for p in paths]
 
     final_path = base_path
     for path in paths: