​

OCA Infrastructure Overview�

​

​

OCA is big

Goals of the OCA infrastructure

• OCA is vast :
• 200 repos
• Hundreds of contributors
• Thousands of modules
• Support the contribution process
• Ensure a baseline quality level with automated checks
• Lower the contribution complexity
• Help ensure consistency across repos
• Promote visibility
• Make OCA addons easy to install for diverse user populations
• … while keeping infrastructure complexity under control

Moving parts...

GitHub.com

translation.�odoo-community.org (weblate)

GitHub actions

pre-commit

tests

click-odoo-makepot

oca-github-bot

pypi.org

wheelhouse.�odoo-community.org

runboat.�odoo-community.org

odoo-community.org Odoo instance

interface-�github

/apps

PSC, CLA

.po

.pot

.pot

pr/push events

pr/push events

pr/push/�comment events

Merge, readme updates, ...

publish

publish

team sync,

cla checks

codecov

Coverage comments

pr events

Coverage report

Parts

• Linting
• Testing
• Using GitHub actions
• Runboot
• The OCA GitHub bot
• Translation
• Publishing
• Teams and CLA

Linting

• Basic code quality
• Catch common errors
• Mostly based on static code analysis
• Automatic formatting of Python, XML and Javascript code
• Opinionated...
• … so reviewers and contributors don’t have to be
• Concentrate on the important stuff
• Automatic corrections of boring stuff (line endings, end-of-file, etc)

​

Linting (<= 12.0)

• Two linters: flake8, pylint(-odoo)
• Located in OCA/maintainer-quality-tools repo
• Served us well over many years
• Easy to update in a central place for all repos
• Harder to keep compatible across Odoo versions
• Complex to configure and run locally
• Not easy to discover their configuration
• Not easy to integrate in IDEs

​

Linting (>= 13.0)

• Using pre-commit (https://pre-commit.com/)
• Many linters available
• Local configuration in each repo easily discoverable
• Easy to run locally and in CI with identical configuration
• Easy to integrate in IDEs (pycharm, vscode, ...)
• Managed centrally with a copier template (https://github.com/OCA/oca-addons-repo-template), �scripted deployment to all repos

​

Linting - pre-commit config

• .pre-commit-config.yaml in each repo
• Each check is called a pre-commit “hook”
• linter configurations in each repo (.flake8, .pylintrc, .eslintrc, ...)
• Generated from OCA/oca-addons-repo-template !

​

• Example: black python code formatter

​

Hook repo �(look there for documentation)

Hook version�(pinned so everyone is on the same page)

Hook to run�(a repo can contain several hooks)

Linting - running locally

• Install pre-commit
• pip3 install --user pre-commit or pipx install pre-commit
• Run manually on all files
• pre-commit run --all
• Run on commit and before push
• pre-commit install at repo root
• Will install a git pre-commit hook so you don’t forget to run it before pushing
• Guaranteed to run the exact same checks as others and CI

Linting - running in CI

GitHub.com

GitHub actions

pre-commit

pr/push events

Linting - interpret output

A check (black in this cased) failed, read output carefully

Linting - interpret output

If you see a diff, run pre-commit locally and git commit all changes.

Just read the instructions :)

Linting - interpret output

Sometimes you need to fix manually

Testing on Travis

GitHub.com

travis-ci.com

GitHub actions

tests

runbot.�odoo-community.org

pr/push events

pr/push events

Testing on Travis

• https://github.com/OCA/maintainer-quality-tools
• Two important scripts
• travis_install_nightly
• travis_run_tests

Testing on Travis travis_install_nightly

• git clone Odoo
• Install Odoo and it’s runtime dependencies
• Dependencies mostly come from Odoo’s requirements.txt
• Install addons to test and their dependencies
• Two modes available: OCA and PIP

​

Testing on Travis�dependencies “OCA” mode

• OCA mode (MQT_DEP=OCA, the default)
• Install OCA/maintainer-quality-tools requirements.txt
• Install requirements.txt from repo
• Parse oca_dependencies.txt for dependent repositories
• Clone each dependent repo
• Install it’s requirements.txt
• Add it to addons path
• Recurse

Testing on Travis

dependencies “OCA” mode

Adding unmerged dependencies temporarily

• in oca_dependencies.txt
• Add reference to repo branch
• {repo} https://github.com/{org}/{repo} {branch}
• Remove branch reference before merging

Testing on Travis

dependencies “PIP” mode

• PIP mode (MQT_DEP=PIP, experimental)
• For each addon in repo
• Start from test-requirements.txt
• For each addon, append this to test-requirements.txt
• -e file://${PWD}/setup/{addon}#egg=odoo14-addon-{addon}
• pip install -r test-requirements.txt --extra-index-url https://wheelhouse.odoo-community.org/oca-simple
• Dependent OCA addons and python libs obtained from published wheels, in turn generated from addons manifest

Testing on Travis

dependencies “PIP” mode

Adding unmerged dependencies

• In test-requirements.txt
• Add pip git reference to addon:
• odoo14-addon-{addon} @ git+https://github.com/OCA/{repo}@refs/pull/{NNN}/head#subdirectory=setup/{addon}
• odoo-addon-{addon} for Odoo >= 15
• Keep PR in Draft status while test-requirements.txt contains references to unmerged addons to avoid accidental merge

Testing on Travis

running tests

• travis_run_tests, test_server.py in OCA/maintainer-quality-tools
• Create a test database
• Detect and install dependent addons
• Install repo addons with --test-enable
• Detect errors in Odoo log file
• What to test is configurable via env vars in .travis.yml
• Default: all addons in repo
• EXCLUDE=”addon1,addon2”: test all but addon1, addon2
• INCLUDE=”addon1,addon2”: test only addon1 and addon2
• UNIT_TEST=1: tests addon one by one (slow)

​

Testing on Travis

More information

See README in OCA/maintainer-quality-tools.

​

See configuration options in OCA/oca-addons-repo-template (.travis.yml.jinja in particular).

Testing on GitHub Actions

Container images

• oca-ci container images
• https://github.com/OCA/oca-ci
• Independent of the CI runners
• Odoo pre-installed
• Built nightly with the latest Odoo and OCB branches
• Example: docker pull ghcr.io/oca/py3.8-odoo15.0

​

Testing on GitHub Actions

• docker pull ghcr.io/oca/py3.8-odoo15.0�
• pip install {dependencies}
• manifestoo check-licenses
• manifestoo check-dev-status
• odoo -i {dependencies}
• odoo -i --test-enable {addons}
• click-odoo-makepot

Testing on GitHub Actions

Each step a single call inside the container.

​

Easy to transition to another CI or self hosted runners if needed.

Testing on GitHub actions

dependencies

• Start from test-requirements.txt
• For each addon, append this to test-requirements.txt
• -e file://${PWD}/setup/{addon}#egg=odoo14-addon-{addon}
• pip install -r test-requirements.txt --extra-index-url https://wheelhouse.odoo-community.org/oca-simple
• Dependent OCA addons and python libs obtained from published wheels, in turn generated from addons manifest

Testing on GitHub actions

dependencies

Adding unmerged dependencies

• In test-requirements.txt
• Add pip git reference to addon:
• odoo14-addon-{addon} @ git+https://github.com/OCA/{repo}@refs/pull/{NNN}/head#subdirectory=setup/{addon}
• odoo-addon-{addon} for Odoo >= 15
• A CI check prevents merging if there are git dependencies in test-requirements.txt

Testing - runboat

• Can launch instance from GitHub PR

​

​

• Runs the same docker images as the CI tests
• View all builds of a repo: runboat.odoo-community.org/builds?repo=oca/mis-builder

Testing - runboat

• View all builds of a repo: https://runboat.odoo-community.org/builds?repo=oca/mis-builder

​

​

​

​

​

​

​

​

​

• Start, stop, reset (re-install).
• Logs: initialization/installation log, running log
• Mail catcher

​

1 row per branch/PR

Code coverage

GitHub.com

travis-ci.com

GitHub Actions

tests

pr/push events

codecov

Coverage comments

Coverage report

Translations

GitHub.com

translation.�odoo-community.org (weblate)

travis-ci.com

GitHub actions

tests

click-odoo-makepot

.po

.pot

.pot

pr/push events

Architecture document

Translations - generate .pot files

• .pot files (.po templates) generated in travis
• when test succeed
• for jobs with MAKEPOT=1 only
• Uses click-odoo-makepot command https://github.com/acsone/click-odoo-contrib/ (wraps the Odoo translation export wizard)
• Pushes changes to .pot files to GitHub
• Needs GitHub token configured in each travis repo

​

Translations - weblate

• Globally configured webhook
• Weblate syncs GitHub repo (rebase)
• Loads modified .pot files
• runs msgmerge -U to update .po files with new terms
• When translators save translations
• Changes to .po files committed to weblate’s git clone, preserving translator login as git commit author
• Periodic push back to GitHub
• Important: only modify .po files in weblate, not in GitHub, to avoid merge conflicts when Weblate syncs

OCA GitHub Bot

GitHub.com

oca-github-bot

pr/push/�comment events

Merge, readme updates, ...

https://github.com/oca/oca-github-bot

OCA GitHub Bot main features

• /ocabot merge command
• Maintenance of the main branch
• (on push to main branch + periodic in case ocabot merge was not used)
• Update addons table in repo-level README.md
• Generate README.rst (oca-gen-addon-readme from OCA/maintainer-tools)
• Generate default addons icons (oca-gen-addon-icon from OCA/maintainer-tools)
• Make default setup.py (also done as pre-commit hook)
• Mention maintainers
• Tag green branches with needs review label
• Tag approved branches (2 approvers) with approved label

​

OCA GitHub Bot - /ocabot merge

• Check if user can merge (push access to repo or declared maintainer in manifest)
• Create a temporary branch off the main branch
• Merge PR into that branch, and push
• Wait for GitHub status notifications
• When all green (ignores runbot and codecov status)
• Check that main branch has not changed (otherwise risk of red branch after merge)
• Run main branch actions (README generator, etc)
• Bump version (patch, minor, major) unless nobump selected
• Build and publish wheel
• Say thank you :)

Publishing

GitHub.com

oca-github-bot

pypi.org

wheelhouse.�odoo-community.org

odoo-community.org Odoo instance

interface-�github

/apps

Merge, readme updates, ...

publish

publish

https://github.com/oca/interface-github

https://github.com/OCA/apps-store

https://github.com/acsone/setuptools-odoo

Teams and CLA

• Listen to PR events to check CLA
• Periodical sync of GitHub teams according to PSC membership

GitHub.com

odoo-community.org Odoo instance

PSC, CLA

team sync,

cla checks

pr events

Monitoring

Contributors Welcome