Skip to content

Commit f2052c5

Browse files
JefffreyJefffrey
authored
[57_maintenance] pull in updated release instructions from main (#11842)
Just so we're consistent with our release process for these backports
1 parent d699f2c commit f2052c5

6 files changed

Lines changed: 105 additions & 326 deletions

File tree

‎dev/release/README.md‎

Lines changed: 31 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ This file documents the release process for the "Rust Arrow Crates": `arrow`, `a
2828
The Rust Arrow Crates are interconnected (e.g. `parquet` has an optional dependency on `arrow`) so we increment and release all of them together.
2929

3030
If any code has been merged to main that has a breaking API change, as defined
31-
in [Rust RFC 1105] he major version number is incremented (e.g. `9.0.2` to `10.0.2`).
31+
in [Rust RFC 1105] the major version number is incremented (e.g. `9.0.2` to `10.0.0`).
3232
Otherwise the new minor version incremented (e.g. `9.0.2` to `9.1.0`).
3333

3434
[rust rfc 1105]: https://github.com/rust-lang/rfcs/blob/master/text/1105-api-evolution.md
@@ -45,56 +45,50 @@ crates.io, the Rust ecosystem's package manager.
4545

4646
We create a `CHANGELOG.md` so our users know what has been changed between releases.
4747

48-
The CHANGELOG is created automatically using
49-
[update_change_log.sh](https://github.com/apache/arrow-rs/blob/main/dev/release/update_change_log.sh)
48+
## Prepare CHANGELOG and version
5049

51-
This script creates a changelog using github issues and the
52-
labels associated with them.
53-
54-
## Prepare CHANGELOG and version:
50+
- Ensure [`git-cliff`](https://git-cliff.org/docs/installation/) is installed
5551

5652
Now prepare a PR to update `CHANGELOG.md` and versions on `main` to reflect the planned release.
5753

58-
Do this in the root of this repository. For example [#2323](https://github.com/apache/arrow-rs/pull/2323)
54+
First copy the contents of `CHANGELOG.md` into `CHANGELOG-old.md`.
55+
56+
Then do this in the root of this repository. For example [#2323](https://github.com/apache/arrow-rs/pull/2323)
5957

6058
```bash
6159
git checkout main
6260
git pull
6361
git checkout -b <RELEASE_BRANCH>
6462

65-
# Update versions. Make sure to run it before the next step since we do not want CHANGELOG-old.md affected.
66-
sed -i '' -e 's/14.0.0/39.0.0/g' `find . -name 'Cargo.toml' -or -name '*.md' | grep -v CHANGELOG.md | grep -v README.md`
63+
# Update versions.
64+
sed -i '' -e 's/14.0.0/39.0.0/g' `find . -name 'Cargo.toml' -or -name '*.md' | grep -v CHANGELOG.md | grep -v CHANGELOG-old.md | grep -v README.md`
65+
cargo check # update Cargo.lock with new versions
6766
git commit -a -m 'Update version'
6867

69-
# ensure your github token is available
70-
export ARROW_GITHUB_API_TOKEN=<TOKEN>
68+
# Assuming remote name is apache; if named differently ensure this is changed,
69+
# since we need the tags from the GitHub repository
70+
git fetch apache --tags
7171

72-
# manually edit ./dev/release/update_change_log.sh to reflect the release version
73-
# create the changelog
74-
./dev/release/update_change_log.sh
75-
# commit the initial changes
76-
git commit -a -m 'Create changelog'
72+
# ensure your github token is available
73+
export GITHUB_TOKEN=<TOKEN>
7774

78-
# run automated script to copy labels to issues based on referenced PRs
79-
# (NOTE 1: this must be done by a committer / other who has
80-
# write access to the repository)
81-
#
82-
# NOTE 2: this must be done after creating the initial CHANGELOG file
83-
python dev/release/label_issues.py
75+
# e.g. TAG=60.0.0; this is just used to format the CHANGELOG, so tag doesn't
76+
# need to exist yet, and don't put an RC tag either
77+
# (we are excluding our above version bump commit from the changelog)
78+
git-cliff --tag <TAG> --unreleased --output CHANGELOG.md --skip-commit $(git rev-parse HEAD)
8479

85-
# review change log / edit issues and labels if needed, rerun, repeat as necessary
86-
# note you need to revert changes to CHANGELOG-old.md if you want to rerun the script
87-
./dev/release/update_change_log.sh
80+
# review change log, adjust labels on PR's as required
81+
# can rerun above git-cliff command if needed
8882

89-
# Commit the changes
90-
git commit -a -m 'Update changelog'
83+
# commit the changelog
84+
git commit -a -m 'Create changelog'
9185

9286
git push
9387
```
9488

9589
Note that when reviewing the change log, rather than editing the
96-
`CHANGELOG.md`, it is preferred to update the issues and their labels
97-
(e.g. add `invalid` label to exclude them from release notes)
90+
`CHANGELOG.md`, it is preferred to update the PRs and their labels
91+
(e.g. add `development-process` label to exclude them from release notes)
9892

9993
Merge this PR to `main` prior to the next step.
10094

@@ -109,7 +103,7 @@ distribution servers.
109103

110104
Pick numbers in sequential order, with `1` for `rc1`, `2` for `rc2`, etc.
111105

112-
### Create git tag for the release:
106+
### Create git tag for the release
113107

114108
While the official release artifact is a signed tarball, we also tag the commit it was created for convenience and code archaeology.
115109

@@ -141,39 +135,12 @@ The `create-tarball.sh` script
141135
apache distribution svn server
142136

143137
2. provide you an email template to
144-
send to dev@arrow.apache.org for release voting.
138+
send to <dev@arrow.apache.org> for release voting.
145139

146140
### Vote on Release Candidate tarball
147141

148-
Send an email, based on the output from the script to dev@arrow.apache.org. The email should look like
149-
150-
```
151-
To: dev@arrow.apache.org
152-
Subject: [VOTE][RUST] Release Apache Arrow
153-
154-
Hi,
155-
156-
I would like to propose a release of Apache Arrow Rust
157-
Implementation, version 4.1.0.
158-
159-
This release candidate is based on commit: a5dd428f57e62db20a945e8b1895de91405958c4 [1]
160-
161-
The proposed release tarball and signatures are hosted at [2].
162-
The changelog is located at [3].
163-
164-
Please download, verify checksums and signatures, run the unit tests,
165-
and vote on the release.
166-
167-
The vote will be open for at least 72 hours.
168-
169-
[ ] +1 Release this as Apache Arrow Rust
170-
[ ] +0
171-
[ ] -1 Do not release this as Apache Arrow Rust because...
172-
173-
[1]: https://github.com/apache/arrow-rs/tree/a5dd428f57e62db20a945e8b1895de91405958c4
174-
[2]: https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-rs-4.1.0
175-
[3]: https://github.com/apache/arrow-rs/blob/a5dd428f57e62db20a945e8b1895de91405958c4/CHANGELOG.md
176-
```
142+
Send an email, based on the output from the script to <dev@arrow.apache.org>.
143+
See an [example of how the email should look](https://lists.apache.org/thread/2vpxdt6n7kzo72sxpr7q8yyby4495gnk).
177144

178145
For the release to become "official" it needs at least three Apache Arrow PMC members to vote +1 on it.
179146

@@ -189,7 +156,7 @@ The `dev/release/verify-release-candidate.sh` script in this repository can assi
189156

190157
If the release is not approved, fix whatever the problem is and try again with the next RC number
191158

192-
### If the release is approved,
159+
### If the release is approved
193160

194161
Then, create a new release on GitHub using the tag `<version>` (e.g. `4.1.0`).
195162

@@ -200,7 +167,7 @@ git tag <version> <version>-<rc>
200167
git push apache <version>
201168
```
202169

203-
Move tarball to the release location in SVN, e.g. https://dist.apache.org/repos/dist/release/arrow/arrow-rs-4.1.0/, using the `release-tarball.sh` script:
170+
Move tarball to the release location in SVN, e.g. <https://dist.apache.org/repos/dist/release/arrow/arrow-rs-4.1.0/>, using the `release-tarball.sh` script:
204171

205172
```shell
206173
./dev/release/release-tarball.sh 4.1.0 2
@@ -212,7 +179,7 @@ Congratulations! The release is now official!
212179

213180
The [`release.yml`] workflow automatically creates a github release for the tag.
214181
Check that the release is created and contains the correct changelog here:
215-
https://github.com/apache/arrow-rs/releases
182+
<https://github.com/apache/arrow-rs/releases>
216183

217184
[`release.yml`]: https://github.com/apache/arrow-rs/blob/main/.github/workflows/release.yml#L1-L0
218185

@@ -249,7 +216,6 @@ Rust Arrow Crates:
249216
(cd arrow-ipc && cargo publish)
250217
(cd arrow-csv && cargo publish)
251218
(cd arrow-json && cargo publish)
252-
(cd arrow-avro && cargo publish)
253219
(cd arrow-arith && cargo publish)
254220
(cd arrow-string && cargo publish)
255221
(cd arrow-row && cargo publish)

‎dev/release/create-tarball.sh‎

Lines changed: 38 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -65,8 +65,9 @@ else
6565
tar=tar
6666
fi
6767

68-
if ! git -C "${SOURCE_TOP_DIR}" rev-list --max-count=1 ${tag}; then
68+
if ! commit_sha=$(git -C "${SOURCE_TOP_DIR}" rev-list --max-count=1 "${tag}"); then
6969
echo "Cannot continue: unknown git tag: $tag"
70+
exit 1
7071
fi
7172

7273

@@ -78,6 +79,33 @@ url="https://dist.apache.org/repos/dist/dev/arrow/${release}-rc${rc}"
7879

7980
echo "Attempting to create ${tarball} from tag ${tag}"
8081

82+
# Write out the submodule commits referenced so that release verification
83+
# can later ensure it uses the same versions of test data.
84+
tmpdir=$(mktemp -d)
85+
trap 'rm -r "${tmpdir}"' EXIT
86+
git -C "${SOURCE_TOP_DIR}" ls-tree -r ${tag} \
87+
| awk '$2 == "commit" { print $4, $3 }' > ${tmpdir}/.gitmodules-commits
88+
89+
# create <tarball> containing the files in git at $tag
90+
# the files in the tarball are prefixed with {release}
91+
# (e.g. apache-arrow-rs-4.0.1)
92+
mkdir -p ${distdir}
93+
(cd "${SOURCE_TOP_DIR}" && \
94+
git archive --prefix ${release}/ \
95+
--add-file ${tmpdir}/.gitmodules-commits ${tag} \
96+
| gzip > ${tarball})
97+
98+
echo "Running rat license checker on ${tarball}"
99+
${SOURCE_DIR}/run-rat.sh ${tarball}
100+
101+
echo "Signing tarball and creating checksums"
102+
gpg --armor --output ${tarball}.asc --detach-sig ${tarball}
103+
# create signing with relative path of tarball
104+
# so that they can be verified with a command such as
105+
# shasum --check apache-arrow-rs-4.1.0-rc2.tar.gz.sha512
106+
(cd ${distdir} && shasum -a 256 ${tarname}) > ${tarball}.sha256
107+
(cd ${distdir} && shasum -a 512 ${tarname}) > ${tarball}.sha512
108+
sha256=$(cut -d ' ' -f 1 "${tarball}.sha256")
81109

82110
echo "Draft email for dev@arrow.apache.org mailing list"
83111
echo ""
@@ -88,14 +116,17 @@ Subject: [VOTE][RUST] Release Apache Arrow Rust ${version} RC${rc}
88116
89117
Hi,
90118
91-
I would like to propose a release of Apache Arrow Rust Implementation, version ${tag}.
119+
I would like to propose a release of Apache Arrow Rust Implementation, version ${version}.
92120
93-
This release candidate is based on commit: ${tag} [1]
121+
This release candidate is based on commit: ${commit_sha} [1].
122+
The SHA256 of the release candidate is: ${sha256}
94123
95124
The proposed release tarball and signatures are hosted at [2].
96125
97126
The changelog is located at [3].
98127
128+
The release tracking issue is: [5]
129+
99130
Please download, verify checksums and signatures, run the unit tests,
100131
and vote on the release. There is a script [4] that automates some of
101132
the verification.
@@ -106,35 +137,15 @@ The vote will be open for at least 72 hours.
106137
[ ] +0
107138
[ ] -1 Do not release this as Apache Arrow Rust ${version} because...
108139
109-
[1]: https://github.com/apache/arrow-rs/tree/${tag}
140+
[1]: https://github.com/apache/arrow-rs/tree/${commit_sha}
110141
[2]: ${url}
111-
[3]: https://github.com/apache/arrow-rs/blob/${tag}/CHANGELOG.md
142+
[3]: https://github.com/apache/arrow-rs/blob/${commit_sha}/CHANGELOG.md
112143
[4]: https://github.com/apache/arrow-rs/blob/master/dev/release/verify-release-candidate.sh
144+
[5]: RELEASE_ISSUE
113145
MAIL
114146
echo "---------------------------------------------------------"
115147

116-
117-
118-
# create <tarball> containing the files in git at $tag
119-
# the files in the tarball are prefixed with {release}
120-
# (e.g. apache-arrow-rs-4.0.1)
121-
mkdir -p ${distdir}
122-
(cd "${SOURCE_TOP_DIR}" && \
123-
git archive ${tag} --prefix ${release}/ \
124-
| gzip > ${tarball})
125-
126-
echo "Running rat license checker on ${tarball}"
127-
${SOURCE_DIR}/run-rat.sh ${tarball}
128-
129-
echo "Signing tarball and creating checksums"
130-
gpg --armor --output ${tarball}.asc --detach-sig ${tarball}
131-
# create signing with relative path of tarball
132-
# so that they can be verified with a command such as
133-
# shasum --check apache-arrow-rs-4.1.0-rc2.tar.gz.sha512
134-
(cd ${distdir} && shasum -a 256 ${tarname}) > ${tarball}.sha256
135-
(cd ${distdir} && shasum -a 512 ${tarname}) > ${tarball}.sha512
136-
137148
echo "Uploading to apache dist/dev to ${url}"
138149
svn co --depth=empty https://dist.apache.org/repos/dist/dev/arrow ${SOURCE_TOP_DIR}/dev/dist
139150
svn add ${distdir}
140-
svn ci -m "Apache Arrow Rust ${version} ${rc}" ${distdir}
151+
svn ci -m "Apache Arrow Rust ${version} ${rc}" ${distdir}

0 commit comments

Comments
 (0)