# Hog-CI with GitHub Actions ```{warning} Outdated Documentation! This documentation version is out of date. Please check the [latest version 2026.1](https://hog.readthedocs.io/en/latest/). ``` :::{toctree} :glob: :hidden: * ::: ## Setting up Hog CI with GitHub Actions On GitHub repository, Hog Continuous Integration makes use of the [GitHub Actions](https://docs.github.com/en/actions) framework. Both the GitHub repository and your local area must be set-up to work properly with Hog CI. In this paragraph, we assume that we are working with a GitHub Project called `MyProject` under the GitHub group `MyGroup`. Please, replace these with the actual names of your project and group. ### Preliminary requirements To run the Hog-CI, you need a GitHub service account, which will actually run the CI. We suggest to create an another account on GitHub.com and give it maintainer permissions. ### Set up your personal GitHub Actions Workflow YAMLs GitHub Actions uses [YAML configuration files](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) to define which commands it must run. These files should be stored inside the `.github/workflows` directory of your repository. Hog cannot provide full YAML files for your project, but templates file can be found under [`Hog` -> `Templates` -> `github-pull.yml` ](https://gitlab.com/hog-cern/Hog/-/blob/master/Templates/github-pull.yml) and [`Hog` -> `Templates` -> `github-main.yml` ](https://gitlab.com/hog-cern/Hog/-/blob/master/Templates/github-main.yml) as a reference. These two files are required to configure the `Merge Request` and `Master` pipelines. #### Pull Request Pipeline YAML configuration For example, suppose we want to write the Yaml configuration files to run the Hog project `my_project` on the CI. This file will actually include the Hog [`hog-pull.yml` configuration file](https://gitlab.com/hog-cern/Hog/-/blob/develop/.github/workflows/Hog-pull.yml), where the CI stages of the *pull pipeline* are defined. An example `.github/workflow/hog-pull.yml` looks like: ```yaml name: Deploy on: pull_request: branches: [ master, main ] jobs: hog-workflow: uses: hog-CERN/Hog/.github/workflows/Hog-pull.yml@Hog2023.1 secrets: SUBMODULE_CONTENT_PULL_KEY: ${{ secrets.SUBMODULE_CONTENT_PULL_KEY }} HOG_PUSH_TOKEN: ${{ secrets.HOG_PUSH_TOKEN }} HOG_EMAIL: ${{ secrets.HOG_EMAIL}} HOG_USER: ${{ secrets.HOG_USER}} EOS_USER: ${{ secrets.EOS_USER }} EOS_PASSWORD: ${{ secrets.EOS_PASSWORD }} HOG_PATH: ${{ secrets.HOG_PATH }} HOG_XIL_LICENSE: ${{ secrets.HOG_XIL_LICENSE }} HOG_IP_PATH: ${{ secrets.HOG_IP_PATH }} HOG_TARGET_BRANCH: ${{ secrets.HOG_TARGET_BRANCH }} HOG_DEVELOP_BRANCH: ${{ secrets.HOG_DEVELOP_BRANCH }} with: BUILD_PROJECTS: >- ["example"] SIM_PROJECTS: >- ["example"] HOG_USE_DOXYGEN: True HOG_CHECK_SYNTAX: True HOG_NO_RESET_BD: True HOG_CHECK_PROJVER: True HOG_NO_BITSTREAM: False HOG_PR_MSG: '1' ``` In the `on: pull_request: ` section you can define the target branches for which you want your CI to run. In the example above, the CI will run for each PR targetting the `master` or the `main` branch of your repository. Here you can select the version of Hog you want to use by changing `Hog2023.1` to your favourite release in the line: ```yaml uses: hog-CERN/Hog/.github/workflows/Hog-pull.yml@Hog2023.1 ``` The version of Hog **MUST** be specified. If you fail to do so, the CI will pick up the parent configuration file from the latest Hog master branch. This is discouraged, since Hog development could lead to not back-compatible changes that could break your CI. Now, you need to define the projects you which to build and/or simulate in the CI. In the code snippet above we are running only the `example` project, but you can run as many projects as you wish, adding them to the list, in the following way: ```yaml with: BUILD_PROJECTS: >- ["proj1", "proj2", "proj3"] SIM_PROJECTS: >- ["proj1", "proj2", "proj3"] ``` ##### Secrets and Inputs A set of environmental variables shall be given to the Hog workflow to operate. These are passed as `inputs` or `secrets`. Input values can be written directly in your YAMLs, since they do not include any sensitive informations, for which we suggest to use GitHub secrets, which can be added in the following way: - Go to https://gitlab.com/MyGroup/MyProject/settings/secrets/actions - Click on `New Repository Secret` button for each variable to add Inputs and secrets can then be defined in your YAML, with the following syntax ```yaml secrets: MYSECRET: ${{ secrets.MYSECRET }} with: MYINPUT: "input_value" ``` This is the list of secrets for the merge request pipeline: | Name | Required | Value | |--------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `SUBMODULE_CONTENT_PULL_KEY` | **YES** | The public key of your account. It is required, otherwise the CI cannot access private submodules. It can be generated following [these instructions](https://samuelsson.dev/access-private-submodules-in-github-actions/). | | `HOG_PUSH_TOKEN` | **YES** | The push token you generated for your service account. It can be generated from https://github.com/settings/tokens, and should have *project*, *repo* and *workflow* rights. | | `HOG_EMAIL` | **YES** | Your service account's email address (e.g. service_account_mail@cern.ch). | | `HOG_USER` | **YES** | Your service account name for GitHub (e.g. `my_service_account`). | | `HOG_PATH` | **YES** | The `PATH` variable for your VM, should include your ISE bin directory. | | `HOG_TCLLIB_PATH` | NO | **MANDATORY for Libero SoC**. Path of installed `tcllib` libraries on your VMs | | `HOG_EXTERNAL_PATH` | NO | If specified, entries in the `.ext` list file would be relative this path | | `HOG_XIL_LICENSE` | NO | Should contain the Xilinx license servers, separated by a comma. You can also use the standard Xilinx variables to set up your licenses.[^license] | | `HOG_LD_LIBRARY_PATH` | NO | If set, the defined path will be added to the default `LD_LIBRARY_PATH` | | `HOG_SIMULATION_LIB_PATH` | NO | The PATH in your VM, where the Simulation Lib files are stored (Vivado only). | | `HOG_IP_PATH` | NO | The path where to store the IP generated results. If not set, the CI will synthesise the IPs each time. If the path starts with `/eos/`, Hog will store the products in the specified EOS storage, otherwise on the running machine. | | `HOG_RESET_FILES` | NO | All the files (or wildcards) contained in this variable (separated by spaces or new lines) be reset at the pre-synthesis time by Hog-CI. | | `EOS_MGM_URL` | NO | Set the EOS instance. If your EOS storage is a user storage use `root://eosuser.cern.ch`. For EOS projects, have a look [here](http://cernbox-manual.web.cern.ch/cernbox-manual/en/project_space/access-to-project-space.html). | | `EOS_USER` | NO | The name of your CERN service account . Only required if you are using the EOS storage cloud space. | | `EOS_PASSWORD` | NO | The password of your CERN service account (should be masked). Only required if you are using the EOS storage cloud space. | | `HOG_TARGET_BRANCH` | NO | Project target branch. Pull request should start from this branch. Default: master. | | `HOG_DEVELOP_BRANCH` | NO | Project develop branch. If defined, pull request should start from this branch. For more information see the [Hog versioning chapter](../02-Hog-Versioning). | | `HOG_APPTAINER_IMAGE` | NO | Path to your `.sif` Apptainer container file, to be used to run the CI. Please, refer to [this chapter](../07a-Containers.md) for more details. | | `HOG_APPTAINER_EXTRA_PATH` | NO | External path to be passed to your Apptainer execution. | And finally, this is the list of inputs: | Name | Required | Value | |--------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `BUILD_PROJECTS` | NO | The list of projects to build. | | `SIM_PROJECTS` | NO | The list of projects to simulate. | | `HOG_USE_DOXYGEN` | NO | Should be set to `True`, if you want the Hog CI to create the Doxygen documentation of your project. | | `HOG_CHECK_PROJVER` | NO | Should be set to `True`, if you want that the Hog CI runs only the projects that have been modified with respect to the target branch | | `HOG_NO_RESET_BD` | NO | If this variable is set to `True`, BD files will NOT be reset at the pre-synthesis time by Hog-CI. | | `HOG_NO_BITSTREAM` | NO | If this variable is set to `True`, Hog-CI runs the implementation but does NOT run the write_bitstream stage | | `HOG_CHECK_SYNTAX` | NO | Should be set to `True`, if you want the Hog CI to check the syntax of your code. | | `HOG_ONLY_SYNTH` | NO | If this variable is set to `True`, Hog-CI will run only the synthesis for all projects in the CI. | | `HOG_PR_MSG` | NO | This variable can be set to 0, 1 (default), 2, 3. It specifies the amount of messages that Hog will write into the pull request page. If set to 0 Hog will not write any message. If set to 1, a single message with a version and resource summary for all the built project is written. If set to 2, a short message declaring that the pipeline is passed is written. If set to 2 Hog will write one message per successful project. | | `HOG_NJOBS` | NO | Number of CPU jobs for the synthesis and implementation. Default: 4. | | `HOG_IP_NJOBS` | NO | Number of CPU jobs for the synthesis and implementation. Default: 4. | #### Main pipeline YAML Configuration In addition, you should also have a configuration file for the *main* pipeline. `.github/workflow/hog-main.yml` looks like: ```yaml name: Deploy on: push: branches: - master - main - release - develop jobs: hog-workflow: uses: hog-CERN/Hog/.github/workflows/Hog-main.yml@Hog2023.1 secrets: SUBMODULE_CONTENT_PULL_KEY: ${{ secrets.SUBMODULE_CONTENT_PULL_KEY }} HOG_PUSH_TOKEN: ${{ secrets.HOG_PUSH_TOKEN }} HOG_EMAIL: ${{ secrets.HOG_EMAIL}} HOG_USER: ${{ secrets.HOG_USER}} EOS_USER: ${{ secrets.EOS_USER}} EOS_PASSWORD: ${{ secrets.EOS_PASSWORD}} with: HOG_CREATE_OFFICIAL_RELEASE: True HOG_OFFICIAL_BIN_PATH: ${{ secrets.HOG_OFFICIAL_BIN_PATH }} ``` Here, you can define the branches for which to run the *main* pipeline in the `on: push` section: ```yaml on: push: branches: - master - main - release - develop ``` Similarly to the pull request pipeline, also the main pipeline requires a list of inputs and secrets to operate. ##### Main pipeline Secrets | Name | Required | Value | |--------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `SUBMODULE_CONTENT_PULL_KEY` | **YES** | The public key of your account. It is required, otherwise the CI cannot access private submodules. It can be generated following [these instructions](https://samuelsson.dev/access-private-submodules-in-github-actions/). | | `HOG_PUSH_TOKEN` | **YES** | The push token you generated for your service account. It can be generated from https://github.com/settings/tokens, and should have *project*, *repo* and *workflow* rights. | | `HOG_EMAIL` | **YES** | Your service account's email address (e.g. service_account_mail@cern.ch). | | `HOG_USER` | **YES** | Your service account name for GitHub (e.g. `my_service_account`). | | `HOG_OFFICIAL_BIN_PATH` | NO | The path for archiving the official binary files of your project. If it starts with `/eos`, it will copy the binary files into the CERN EOS storage, otherwise on the local machine. | | `EOS_MGM_URL` | NO | Set the EOS instance. If your EOS storage is a user storage use `root://eosuser.cern.ch`. For EOS projects, have a look [here](http://cernbox-manual.web.cern.ch/cernbox-manual/en/project_space/access-to-project-space.html). | | `EOS_USER` | NO | The name of your CERN service account . Only required if you are using the EOS storage cloud space. | | `EOS_PASSWORD` | NO | The password of your CERN service account (should be masked). Only required if you are using the EOS storage cloud space. | ##### Main pipeline Inputs | Name | Required | Value | |--------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `HOG_CREATE_OFFICIAL_RELEASE` | NO | If this variable is set to `True`, Hog-CI creates an official release note using the version and timing summaries taken from the artefact of the projects. | | `HOG_USE_DOXYGEN` | NO | Should be set to `True`, if you want the Hog CI to create the Doxygen documentation of your project | #### GitHub Repository configuration ### Merge Commit handling on GitHub.com Unfortunately, on GitHub.com is not possible to merge a pull-request with a fast-forward method. Hog emulates this behaviour by deleting the merge commit, when the pull request is merged. For this reason, the repository must be set up in the following way - Go to https://github.com/MyGroup/MyProject/settings - Go to the __Pull Request__ section - Select *Allow merge commits* with *Default message* ### GitHub actions configuration We need to allow the repository to use the reusable workflow provided by Hog. - Go to https://gitlab.com/MyGroup/MyProject/settings/actions - Go to __Action permissions__ - Select *Allow all actions and reusable workflow* The CI shall have write to permissions to create tag and remove merge commits. This can be set up in the following way. - https://gitlab.com/MyGroup/MyProject/settings/actions - Go to __Workflow permissions__ - Select *Read and write permissions* ### Set-up Runners Unfortunately, we cannot use shared runners for every CI job, as the necessary software (Xilinx Vivado, Mentor Graphics Questasim, etc.) are not available. The download, installation and licensing processes would have to be done at each time that the CI is started, slowing down the entire process. As a consequence, you need to set-up your own physical or virtual machines. To set up your machine as self-hosted GitHub runner for your repository, you should: - Go to https://gitlab.com/MyGroup/MyProject/settings/actions/runners - Click on the `New self-hosted runner` button - Follow the instructions for your system Remember that to run Hog, in addition to your chosen IDE, a recent version of git (>2.9.3) must be installed on your private machine. Once the GitHub runner is successfully installed, it will appear in the runners section as in the following screenshot. ### Archive binary files on EOS space (Optional) The Hog CI will produce some artefacts. These include the resulting binary files of your firmware projects and, optionally, the Doxygen documentation html files. Hog has also the ability to copy these files into a desired folder, either on the local build machine or, if working at CERN, on the EOS cloud storage. To enable this feature, we have to specify the GitHub `HOG_OFFICIAL_BIN_PATH` secret. If this variable starts with `/eos`, Hog will understand that it is a EOS folder and will copy the binary there using the [`eos` client software](https://eos-docs.web.cern.ch/diopside/manual/hardware-installation.html#installation), which must be installed on your local machine. To work, you need also to setup the `EOS_MGM_URL` secret accordingly. If you wish to have your files to be accessible in a web browser, you should create a web page in EOS, following [these instructions](http://cernbox-manual.web.cern.ch/cernbox-manual/en/web/). For a personal project, by default, the website will be stored in `/eos/user///www`. The Hog EOS paths must be then sub-folders of the website root path. To expose the files in the website, follow these [instructions](http://cernbox-manual.web.cern.ch/cernbox-manual/en/web/expose_files_in_website.html). ### Speed up the IP synthesis (Optional) In order to save resources and possibly speed up the workflow, the products of the IP synthesis can be copied into an external directory, and retried before starting the firmware synthesis. The external path is specified via the `HOG_IP_PATH` variable. The IPs are archived in the IP path using the md5 hash of the `.xci` file, so if an IP is modified, it will be copied again. It is recommended to clean the IP path regularly as Hog will not delete old IP products. :::{admonition} Storing IPs on EOS Staring from the version 2022.1, Hog automatically detects if the IP storage path is on EOS or on the running machine, verifying that the path starts with `/eos/` or not. :::