ONNX Releases

The ONNX project, going forward, will plan to release roughly on a four month cadence. We follow the Semver versioning approach and will make decisions as a community on a release by release basis on whether to do a major or minor release.

Preparation

  • Determine version (X.Y.Z) for the new release

    • Discuss in Slack channel for Releases (https://lfaifoundation.slack.com/archives/C018VGGJUGK)

    • For (v.X.Y.Z), if release is to be 1.16.0,

      • X=1, Y=16, Z=0

      • The new branch will be rel-1.16.0

        • Branch protections rules are automatically applied to branches following this format.

      • The new tag will be v1.16.0

  • Create new page for the release in Release logistics wiki

Create Release Branch

  • In main branch, before creating the release branch:

    1. Bump the LAST_RELEASE_VERSION in version.h.

      • Set to X.Y.Z, which is same as the release branch you are currently creating.

      • After the release branch is cut, VERSION_NUMBER in main will be increased to the next future version.

    2. Make sure the release version, IR version, ai.onnx opset version, ai.onnx.ml opset version, and ai.onnx.training opset version are correct for the new release in ONNX proto files, Versioning.md, schema.h, helper.py, and helper_test.py.

  • Create a release branch

    1. Click “New branch” from branches and choose main as Source.

    2. Make sure all tests pass on the new branch.

  • After cutting the release branch:

    1. Create PR to set VERSION_NUMBER file in main to the next future releas, X.Y+1.0.

    2. Create PR to set VERSION_NUMBER file in the new release’s branch to X.Y.Zrc1.

      • For example the first release candidate for 1.16.0 would be 1.16.0rc1

    3. Bump opset version for ai.onnx domain in onnx/defs/operator_sets.h and onnx/defs/schema.h for use by future operator additions and changes.

Upload release candidate to TestPyPI

Important

  • WAIT for PR to set the release branch’s VERSION_NUMBER to merge and build before continuing.

  • To push files to TestPyPI or PyPI, install twine if you don’t already have it: pip install twine

    • When prompted for a password by twine commands, use an API token. Your password will not work.

  • Like PyPI, A release version can only be pushed to TestPyPI ONCE.

    • To update an already pushed file, you must increase the VERSION_NUMBER, rebuild, and push a new X.Y.Zrc2, etc.

    • To test push commands, you can use docker or podman to create a local pypi server

      1. Start server docker run --rm -it --platform linux/amd64 -p 80:8080 pypiserver/pypiserver:latest run -a . -P .

        • This starts a local pypiserver that does not require authentication (any user/password will work on it).

        • The container does not save state. Stopping and starting it again will let you push the same version multiple times.

      2. To push files:

        • wheels: twine upload --repository-url http://127.0.0.1:80 --verbose -u fake -p fake *.whl

        • source: twine upload --repository-url http://127.0.0.1:80 --verbose -u fake -p fake dist/*

      3. To pull and install from your test server:

        • pip uninstall -y onnx && pip install --index-url http://127.0.0.1:80/simple/ --pre onnx

Push Wheels

  1. Gather the wheel files from the ONNX Github Actions for the release candidate.

    • For each ONNX GitHub Action:

      • ONNX GitHub Action

      • Find the run for the release branch

        • Or start a run by clicking “Run workflow”, pick the release branch, Click “Run Workflow”

      • Click the completed run, scroll to the “Artifacts” section (bottom), and click “wheels” to download the files

      • Extract the wheels.zip files and combine their contents into a single folder

  2. Upload the produced wheels manually to TestPyPI: twine upload --repository testpypi --verbose -u <YOUR_TESTPYPI_USER> <extracted_wheel.zip_folder>/*.whl.

    • A current owner of the ONNX project will need to give you access to the project before you can push files.

    • The project name and version built into the files.

Source Distribution

  1. Make sure all the git submodules are updated

    • git submodule update --init

  2. Make sure the git checkout is clean –

    • Run git clean -nxd

      • Make sure that none of the auto-generated header files such as the following are present.

        • onnx/onnx-operators.pb.cc

        • onnx/onnx-operator.pb.h

        • onnx/onnx.pb.cc

        • onnx/onnx.pb.h

      • If they are present run git clean -ixd and remove those files from your local branch

  3. Generate the source distribution file: python -m build --sdist

    • Run pip install build if you don’t already have the build package.

  4. Upload source distribution file to TestPyPI: twine upload --repository testpypi --verbose -u <YOUR_TESTPYPI_USER> dist/*

    • Notes:

      • A current owner of the ONNX project will need to give you access to the project before you can push files.

      • The project name and version built into the files.

  5. Confirm TestPyPI package can be installed:

    • Wheel install: pip uninstall -y onnx && pip install -i https://test.pypi.org/simple/ --pre onnx

      • Assumes pre-built while is available for your environment, if not a source install will start.

    • Source install: pip uninstall -y onnx && pip install -i https://test.pypi.org/simple --no-binary onnx --pre onnx

Package verification

Test ONNX itself

  • Test the PyPI package installation with different combinations of various Python versions, Protobuf versions and platforms.

    • After installing the TestPyPI package, run pytest in the release branch.

    • Python versions : Applicable python versions for the release.

    • Protobuf versions : Latest protobuf version at the time of the release + protobuf version used for previous release

Partner Validation

Official Release

Validation steps must be completed before this point! This is the point of new return.

  • git tags should not be changed once published

  • Once pushed to PyPI there is no way to update the release. A new release must be made instead

Set final version number

  • Create PR to remove “rcX” suffix from VERSION_NUMBER file in the new release’s branch.

Create release tag

  • Draft a release based on the release branch:

    • DO NOT click Publish release until you are sure no more changes are needed.

      • Use Save Draft if need to save and update more later.

      • Publishing will create the new git tag

    • Tag: See top of Preparation for tag to create.

    • Target: The release branch that was just cut

    • Previous tag: Select the previous release.

    • Write:

      • Use previous releases as a template

      • Use information from Release logistics wiki which should have been created prior to branch cut.

      • Add any additional commits that merged into the release in since wiki was written.

    • .tar.gz and .zip will be auto-generated after publishing the release.

Upload to Official PyPI

NOTES:

  • Once the packages are uploaded to PyPI, you cannot overwrite it on the same PyPI instance.

    • Please make sure everything is good on TestPyPI before uploading to PyPI**

  • PyPI has separate logins, passwords, and API tokens from TestPyPI but the process is the same. An ONNX PyPI owner will need to grant access, etc.

Follow the Wheels and Source Distribution steps in Upload release candidate toTestPyPI above with the following changes:

  • Create a new API token of onnx scope for uploading onnx wheel in your PyPI account (API tokens section).

    • Remove the created token after pushing the wheels and source for the release.

  • When uploading, remove --repository testpypi from twine commands.

  • When verifying upload, remove -i https://test.pypi.org/simple/ and --pre from pip commands.

After PyPI Release

Announce

Update conda-forge package with the new ONNX version

Conda builds of ONNX are done via conda-forge/onnx-feedstock, which runs infrastructure for building packages and uploading them to conda-forge.

Merge into main branch

  • If urgent changes were made directly into the release branch, merge the release branch back into main branch.

  • If all PRs merged into the release branch (after it was cut) were cherry-picks from main, the merge PR will show as empty and this step is not needed.

Remove old onnx-weekly packages on PyPI

  • Remove all onnx-weekly packages from PyPI for the just released version to save space.

  • Steps:

    • Go to PyPI onnx-weekly/releases

      • This is a separate project than the onnx releases so you may need to request access from an owner

    • Click target package -> Options -> Delete.