How does the Github Classroom get the results from the autograder?

[kenpower]

Hi everyone,

I am trying to create a custom autograder action for a complex series of tests and to give better feedback.

But looking at the official autograder code (https://github.com/education/autograding) I can't see where the points are stored/send somewhere that Github classroom can access. The only output I see from the autograder is to the run check, but that is an unstructured string?

Anyone know how this works? How does Github classroom get the grade?

A suggestion...(optional)

more help with creating custom actions

[kenpower]

I don't know if this will work, or even if it is ok to do, but I am going to tag @markpatterson27 here (contributor to autograding action), maybe they will know the answer?

[markpatterson27]

I think you've answered your own question. Autograding uses check suite to report the grading score to GH classroom. (Specifically, this call: https://octokit.github.io/rest.js/v18#checks-update)

There's an issue with an actions/github-script that pulls the relevant code: github-education-resources/autograding#52 I haven't checked , but change the text string to the value you want (including the 'Points ' prefix in the format Points {points}/{total-points}). Presumably GH classroom parses the string on the other end. Pretty sure the points have to be integers.

I have thought about putting that code into a separate action or proposing a points input on the autograding action, for cases where it's easier to calculate the points outside of the autograding action.

(Also, most of my contributions were to the package-lock.json file. Not all code is equal ;))

[kenpower]

Thanks so much. I assumed that there might be a more "formal" mechanism, i.e. not using a human string to pass numerical data. But having explored more, there are not a lot of other options.

"Presumably GH classroom parses the string on the other end. "

I would like to see that, but AFAIK GH Classroom is now closed source, is that correct?

[markpatterson27]

As an alternative to creating a custom autograder, you could create scripts that do the grading and feedback for you, and just use autograding to call those scripts.

See https://github.com/SERC-COM299/CookieDB-KV1-CREATE-and-INSERT/blob/6bb50fa2e734cd9798da8806811ffc016c11ee20/.github/classroom/autograding.json#L44-L51 for an example of grading if the student has used the correct data type for an attribute in a SQL database (along with providing them feedback as a comment in the 'Feedback' PR).

[kenpower]

I had thought of that, but the problem is the autograder will only give a score if ALL the tests in a script pass. I have a file with 40 unit tests and I want to give a score if say 15 tests pass, the score should be 15/40. Going further, I would like to have different scores for each unit test.

I could set up 40 different blocks in the autograding.json like in your example, but I have to build that and run my test runner 40+ times. Seems like a bit overkill when the test runner can run once and can already report how many tests failed/passed.

Maybe there could be an option for the test runner script to create an env var that the autograder uses (like you suggested above)?

[kenpower]

Thanks for linking the SQL testing script, there is a lot to learn from it!

[markpatterson27]

Have you signed up for the autograding usability study? They're currently looking at ways to make autograding more flexible. Your input would probably be helpful.

As a quick test, I put together direct points input for autograding. If you want to try it out replace the education/autograding step with this:

      - uses: markpatterson27/autograding@dev-points-input-release
        with:
          points: 10
          available-points: 50

• points is the amount of points the student as scored
• available-points is the total amount of available points to earn

You can pass values between steps in a workflow, either by writing to the $GITHUB_ENV file or you can create a step output by writing to the $GITHUB_OUTPUT file. See: https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-environment-variable.

So, the business part of your workflow might look something like

...
      - name: run grading
        id: grading
        run: <command to run unit tests with grading output>

      - uses: markpatterson27/autograding@dev-points-input-release
        with:
          points: ${{ steps.grading.outputs.points }}
          available-points: ${{ steps.grading.outputs.total-points }}

[kenpower]

Thanks you so much for this.
I tried out the new version, it mostly works, but there are a couple of issues (probably on the classroom side and maybe unfixable?).

First, the score is correct on the classroom web page (great!) (not sure why it's saying no commits?)

However, in the results download, it shows a zero for points available:

But that is easily fixable on the teachers side.

Brilliant! What is the next step, will there be a PR to main?

Thanks Again!

[markpatterson27]

I opened a PR for it: github-education-resources/autograding#248

[agilebotanist]

This method works fine for the markpatterson27/autograding@dev-points-input-release alone.

Classroom updates as needed. I hoped to use this as a workaround by combining with the official Autograding. Since currently my Classroom does not update grades labels :/

Here is my code to debug and pass the grades to the Classroom by this custom action. Unfortunately, this workaround does not work.

name: GitHub Classroom Workflow

on: [push]

jobs:
  build:
    permissions: write-all
    name: Autograding
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Auto
        id: auto
        uses: education/autograding@v1
      - name: Out
        id: out
        env: 
          GRADES: ${{join(steps.auto.outputs.*, '\n')}}
        run: |
          echo "Auto print $GRADES"
          points=$(echo "$GRADES" | awk -F'/' '{print $1}')
          echo 'points<<EOF' >> $GITHUB_OUTPUT
          echo $points >> $GITHUB_OUTPUT
          echo EOF >> $GITHUB_OUTPUT
          totalpoints=$(echo "$GRADES" | awk -F'/' '{print $2}')
          echo 'totalpoints<<EOF' >> $GITHUB_OUTPUT
          echo $totalpoints >> $GITHUB_OUTPUT
          echo EOF >> $GITHUB_OUTPUT
      - name: Get points
        env:
          POINTS: ${{ steps.out.outputs.points }}
          TOTAL: ${{ steps.out.outputs.totalpoints }}
        run: echo "Grades $POINTS / $TOTAL"
      - name: mark
        id: mark
        env:
          POINTS: ${{ steps.out.outputs.points }}
          TOTAL: ${{ steps.out.outputs.totalpoints }}
#       run: echo "points=7" >> "$GITHUB_OUTPUT" ; echo "total-points=9" >> "$GITHUB_OUTPUT"
        uses: markpatterson27/autograding@dev-points-input-release
        with:
          points: ${{ steps.out.outputs.points }}
          available-points: ${{ steps.out.outputs.totalpoints }}
#          points: 8
#          available-points: 15

[ProfWentao]

https://github.com/education/autograding no longer exists. So what is the official way to send the test result to github classroom so it can display the score? Do we have to use this instead? markpatterson27/autograding@dev-points-input-release

[theeluwin]

It's now called classroom resources https://github.com/classroom-resources/autograding-grading-reporter

[theeluwin]

(in version of Aug 19, 2025) Specific things must happen: notice, notice title, and content.

Example:

PASS=2
TOTAL=5
echo "::notice title=Autograding complete::Points ${PASS}/${TOTAL}"
echo "::notice title=Autograding report::{\"totalPoints\":${PASS},\"maxPoints\":${TOTAL}}"

We need "Autograding complete" and "Autograding report" as notice titles and exactly that format.
Now we can skip the autograding reporter and use our custom autograding actions.

[C5T8fBt-WY]

I was running into issues when trying @theeluwin’s solution.

According to a PR on the autograding‑grading‑reporter repo, only one check per workflow is parsed reliably. In my case, I use EnricoMi/publish-unit-test-result-action, which creates an additional check. To avoid conflicts, I therefore had to set check_run: false.

[mikaelv]

Could you please share a sample build file ? I'm trying to run Java tests using maven, then grade one point per passed test. It looks like EnricoMi/publish-unit-test-result-action could help me parse the test results, but I'm not sure how to report the grades to classroom.

[C5T8fBt-WY]

I can share a sample workflow file used in a C programming class that utilizes Google Test. The Python script for generating the report is available here.

name: Auto Grader

on:
  push:
  workflow_dispatch:

permissions:
  contents: write
  pull-requests: write

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      # Build and test steps go here

      - name: Publish Test Results
        uses: EnricoMi/publish-unit-test-result-action@v2
        id: test-results
        if: always()
        with:
          files: |
            test-results/**/*.xml
          check_run: false
          comment_title: "Multi-Assignment Test Results"
          json_file: test-results.json
          json_test_case_results: true
          json_suite_details: true
          report_individual_runs: true
          deduplicate_classes_by_file_name: false

      - name: Generate Test Report
        if: always()
        run: |
          python3 scripts/generate_report.py test-results.json > test-report.md
          cat test-report.md

      - name: Post Test Report as Commit Comment
        uses: peter-evans/commit-comment@v3
        if: always()
        with:
          body-path: test-report.md
          token: ${{ secrets.GITHUB_TOKEN }}

      - name: Notify GitHub Classroom of results
        if: always()
        run: |
          PASS=${{ fromJSON(steps.test-results.outputs.json).stats.tests_succ }}
          TOTAL=${{ fromJSON(steps.test-results.outputs.json).stats.tests }}

          echo "::notice title=Autograding complete::Points ${PASS}/${TOTAL}"
          echo "::notice title=Autograding report::{\"totalPoints\":${PASS},\"maxPoints\":${TOTAL}}"

      - name: Fail if tests failed
        if: fromJSON(steps.test-results.outputs.json).conclusion != 'success'
        run: exit 1

[mikaelv]

thanks a lot ! It works well with Java & Maven and saved me a lot of time figuring it out. I'll use it in my classrooms.