welcome: please sign in
location: Diff for "versionControl/canvasSync"
Differences between revisions 6 and 15 (spanning 9 versions)
⇤ ← Revision 6 as of 2020-02-14 08:47:06 →
Size: 3609
Editor: albheim
Comment:
← Revision 15 as of 2020-02-21 14:03:54 → ⇥
Size: 2668
Editor: albheim
Comment:
Deletions are marked like this. Additions are marked like this.
Line 5: Line 5:
For a working example check out the [[https://gitlab.control.lth.se/regler/FRTF10|FRTF10-repo]] where as soon as any changes in pdfs are pushed to the `to_canvas` folder on the master branch they are uploaded to the `from_gitlab` folder on the [[https://canvas.education.lu.se/courses/1342/files/folder/from_gitlab|canvas page]]. For a working example check out the [[https://gitlab.control.lth.se/regler/FRTF10|FRTF10-repo]] where as soon as any changes in pdfs are pushed to the `to_canvas` folder on the master branch they are uploaded to the `to_canvas` folder on the [[https://canvas.education.lu.se/courses/1342/files/folder/to_canvas|canvas page]].
Line 8: Line 8:
Gitlab CI is run through a file called `.gitlab-ci.yml` in the root folder of the repository. This file can setup the environment we need to run certain scripts, define how and what scripts should be run and define in what cases it should be run. For our example we use this file Gitlab CI is run through a file called `.gitlab-ci.yml` in the root folder of the repository. This file can setup the environment we need to run certain scripts, define how and what scripts should be run and define in what cases it should be run. For our example we use [[https://gitlab.control.lth.se/regler/FRTF10/blob/master/.gitlab-ci.yml|this]] file
Line 17: Line 17:
    - python3 to_canvas/canvas_sync.py     - python3 to_canvas.py
Line 25: Line 25:
where `image` tells the runner to use a python3 docker image, `before_script` runs a command in the container created from the image that installs the package canvasapi and `canvas_push` runs a script if any of the specified files has changed for the specified branch. where `image` tells the runner to use a python3 docker image, `before_script` runs a command in the container created from the image that installs the package canvasapi and the python script runs if any pdf-files in the folder to_canvas has been modified.
Line 31: Line 31:
The current script just takes all pdf files in one folder and pushed them to a folder on canvas. The script needs the names of the folders and the course code in canvas. To change how the upload is done you simply need to change this script to find and upload things in another fashion.
{{{#!highlight python
import os
from canvasapi import Canvas		 The [[https://gitlab.control.lth.se/regler/FRTF10/blob/master/to_canvas.py|current script]] uploads all pdf files from a folder (with subfolders) and creates the corresponding folder structure on canvas.
Line 36: Line 33:
# Set these three values
canvas_course_code = 1342
canvas_pdf_path = "from_gitlab"
source_path = "to_canvas"		 The script assumes there exists environment variables for a canvas API token and canvas course code. A token can be added by navigating to https://gitlab.control.lth.se/regler/<REPO>/-/settings/ci_cd, where you replace <REPO> with the repository name, and looking under Variables. There you add a token with name `CANVAS_TOKEN` and one with name `CANVAS_COURSE_CODE`.
Line 41: Line 35:
canvas_base_path = "course files"
API_URL = "https://canvas.education.lu.se/"
API_TOKEN = os.environ["CANVAS_TOKEN".format(canvas_course_code)]

canvas = Canvas(API_URL, API_TOKEN)
course = canvas.get_course(canvas_course_code)

# Check if folder exists, otherwise create it
base_folder = None
pdf_folder = None
for folder in course.get_folders():
    if folder.full_name == "{}/{}".format(canvas_base_path, canvas_pdf_path):
        pdf_folder = folder
    if folder.full_name == canvas_base_path:
        base_folder = folder
if pdf_folder is None:
    pdf_folder = base_folder.create_folder(canvas_pdf_path)

# Find all pdf files in to_canvas and upload to canvas folder from_gitlab
for f in os.listdir(source_path):
    if f.endswith(".pdf"):
        pdf_folder.upload(os.path.join(source_path, f))
}}}		 Currently the script will upload and replace files if they already exists, but it wont otherwise delete files if they are removed from the repo.
Line 66: Line 38:
The script assumes there exists a variable for the Canvas API token in the repository. This variable is added to the gitlab runner as an environment variable and can then be used without us having it hardcoded anywhere. A token can be added by navigating to https://gitlab.control.lth.se/regler/<REPO>/-/settings/ci_cd, where you replace <REPO> with the repository name, and looking under Variables. There you add a token with name `CANVAS_TOKEN` and value `13596~t2TYuTRlaJnfvyTznfvhDOqq7B85o2wD2jU8NBCsixzLirB8nceTFIvUzbldIa1K`.
The API token can also be generated in a canvas profile but this is a token generated from a department account.		 The API token can be generated in a canvas profile, and this one is generated from an account that should have access to all courses run by Automatic Control `13596~t2TYuTRlaJnfvyTznfvhDOqq7B85o2wD2jU8NBCsixzLirB8nceTFIvUzbldIa1K`.

Gitlab to Canvas syncing

For a working example check out the FRTF10-repo where as soon as any changes in pdfs are pushed to the to_canvas folder on the master branch they are uploaded to the to_canvas folder on the canvas page.

Gitlab CI

Gitlab CI is run through a file called .gitlab-ci.yml in the root folder of the repository. This file can setup the environment we need to run certain scripts, define how and what scripts should be run and define in what cases it should be run. For our example we use this file 

   1 image: "python:3.7"
   2 
   3 before_script:
   4   - python3 -m pip install canvasapi
   5 
   6 canvas_push:
   7   script:
   8     - python3 to_canvas.py
   9   stage: deploy
  10   only:
  11     refs:
  12       - master
  13     changes:
  14       - to_canvas/**/*.{pdf}

where image tells the runner to use a python3 docker image, before_script runs a command in the container created from the image that installs the package canvasapi and the python script runs if any pdf-files in the folder to_canvas has been modified.

Runners

See https://gitlab.control.lth.se/regler/<REPO>/-/settings/ci_cd, where <REPO> should be replaced with the repository name, for instructions about runners. Hopefully we will soon have some shared runners for the department, but for the moment they have to be set up for each project specifically.

Python script

The current script uploads all pdf files from a folder (with subfolders) and creates the corresponding folder structure on canvas.

The script assumes there exists environment variables for a canvas API token and canvas course code. A token can be added by navigating to https://gitlab.control.lth.se/regler/<REPO>/-/settings/ci_cd, where you replace <REPO> with the repository name, and looking under Variables. There you add a token with name CANVAS_TOKEN and one with name CANVAS_COURSE_CODE.

Currently the script will upload and replace files if they already exists, but it wont otherwise delete files if they are removed from the repo.

Canvas API Token

The API token can be generated in a canvas profile, and this one is generated from an account that should have access to all courses run by Automatic Control 13596~t2TYuTRlaJnfvyTznfvhDOqq7B85o2wD2jU8NBCsixzLirB8nceTFIvUzbldIa1K.

versionControl/canvasSync (last edited 2021-01-15 16:08:45 by albheim)