|
Size: 3609
Comment:
|
← Revision 24 as of 2021-01-15 16:08:45 ⇥
Size: 2051
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 5: | Line 5: |
| For a working example check out the [[https://gitlab.control.lth.se/regler/FRTF10|FRTF10-repo]] where as soon as any changes in pdfs are pushed to the `to_canvas` folder on the master branch they are uploaded to the `from_gitlab` folder on the [[https://canvas.education.lu.se/courses/1342/files/folder/from_gitlab|canvas page]]. | For a working example check out the [[https://gitlab.control.lth.se/regler/FRTF10|FRTF10-repo]] where as soon as anything is pushed to the `master` branch gitlab will automatically upload everything from the `to_canvas` folder to the `from_git` folder found in the [[https://canvas.education.lu.se/courses/1342/files/folder/from_git|course files]] on canvas. |
| Line 8: | Line 8: |
| Gitlab CI is run through a file called `.gitlab-ci.yml` in the root folder of the repository. This file can setup the environment we need to run certain scripts, define how and what scripts should be run and define in what cases it should be run. For our example we use this file {{{#!highlight yaml image: "python:3.7" before_script: - python3 -m pip install canvasapi canvas_push: script: - python3 to_canvas/canvas_sync.py stage: deploy only: refs: - master changes: - to_canvas/**/*.{pdf} }}} where `image` tells the runner to use a python3 docker image, `before_script` runs a command in the container created from the image that installs the package canvasapi and `canvas_push` runs a script if any of the specified files has changed for the specified branch. |
Gitlab CI is run through a file called `.gitlab-ci.yml` in the root folder of the repository. This file can setup the environment we need to run certain scripts, define how and what scripts should be run and define in what cases it should be run. An example file can be found [[https://gitlab.control.lth.se/regler/canvassync/-/blob/master/gitlab-ci.yml|here]]. It contains the CANVAS_COURSE_CODE variable which allows the script to figure out which canvas page it should upload to. |
| Line 28: | Line 11: |
| See https://gitlab.control.lth.se/regler/<REPO>/-/settings/ci_cd, where <REPO> should be replaced with the repository name, for instructions about runners. Hopefully we will soon have some shared runners for the department, but for the moment they have to be set up for each project specifically. | Runners are computers that clone the repo and runs the scripts once something is pushed to `master`. Currently Anders Nilsson has set up his computer as a runner for GitLab, but it should work with other runners also. |
| Line 31: | Line 15: |
| The current script just takes all pdf files in one folder and pushed them to a folder on canvas. The script needs the names of the folders and the course code in canvas. To change how the upload is done you simply need to change this script to find and upload things in another fashion. {{{#!highlight python import os from canvasapi import Canvas |
The [[https://gitlab.control.lth.se/regler/canvassync/-/blob/master/canvas_sync.py|python script]] doing the work in the background just scans for a folder called `to_canvas` in the root directory and using the `canvasapi` package it uploads each individual file with corresponding file structure into the `from_git` folder in canvas. |
| Line 36: | Line 17: |
| # Set these three values canvas_course_code = 1342 canvas_pdf_path = "from_gitlab" source_path = "to_canvas" canvas_base_path = "course files" API_URL = "https://canvas.education.lu.se/" API_TOKEN = os.environ["CANVAS_TOKEN".format(canvas_course_code)] canvas = Canvas(API_URL, API_TOKEN) course = canvas.get_course(canvas_course_code) # Check if folder exists, otherwise create it base_folder = None pdf_folder = None for folder in course.get_folders(): if folder.full_name == "{}/{}".format(canvas_base_path, canvas_pdf_path): pdf_folder = folder if folder.full_name == canvas_base_path: base_folder = folder if pdf_folder is None: pdf_folder = base_folder.create_folder(canvas_pdf_path) # Find all pdf files in to_canvas and upload to canvas folder from_gitlab for f in os.listdir(source_path): if f.endswith(".pdf"): pdf_folder.upload(os.path.join(source_path, f)) }}} |
Currently the script will only upload files and replace if they already exists, but it wont otherwise delete files if they are removed from gitlab. |
| Line 66: | Line 20: |
| The script assumes there exists a variable for the Canvas API token in the repository. This variable is added to the gitlab runner as an environment variable and can then be used without us having it hardcoded anywhere. A token can be added by navigating to https://gitlab.control.lth.se/regler/<REPO>/-/settings/ci_cd, where you replace <REPO> with the repository name, and looking under Variables. There you add a token with name `CANVAS_TOKEN` and value `13596~t2TYuTRlaJnfvyTznfvhDOqq7B85o2wD2jU8NBCsixzLirB8nceTFIvUzbldIa1K`. The API token can also be generated in a canvas profile but this is a token generated from a department account. |
The API token can be generated in a canvas profile, and this one is generated from an account that should have access to all courses run by Automatic Control. It is available to all repositories in the regler group on gitlab as an environment variable under the name CANVAS_TOKEN. |
Gitlab to Canvas syncing
For a working example check out the FRTF10-repo where as soon as anything is pushed to the master branch gitlab will automatically upload everything from the to_canvas folder to the from_git folder found in the course files on canvas.
Gitlab CI
Gitlab CI is run through a file called .gitlab-ci.yml in the root folder of the repository. This file can setup the environment we need to run certain scripts, define how and what scripts should be run and define in what cases it should be run. An example file can be found here. It contains the CANVAS_COURSE_CODE variable which allows the script to figure out which canvas page it should upload to.
Runners
Runners are computers that clone the repo and runs the scripts once something is pushed to master. Currently Anders Nilsson has set up his computer as a runner for GitLab, but it should work with other runners also.
Python script
The python script doing the work in the background just scans for a folder called to_canvas in the root directory and using the canvasapi package it uploads each individual file with corresponding file structure into the from_git folder in canvas.
Currently the script will only upload files and replace if they already exists, but it wont otherwise delete files if they are removed from gitlab.
Canvas API Token
The API token can be generated in a canvas profile, and this one is generated from an account that should have access to all courses run by Automatic Control. It is available to all repositories in the regler group on gitlab as an environment variable under the name CANVAS_TOKEN.